Setting Up Google Drive With Rclone Cache And Crypt
Rclone with Google drive
General information
rClone is installed by default on all Appboxes, with a version update every 7 days. For more information about rClone please refer to the official rClone documentation.
rClone setup and running requires certain level of knowledge.
Bytesized Hosting does not provide support or will help debugging issues caused by rClone. See this news item for more details on why that is the case.
Purpose of this wiki
This wiki will explain how to connect your Google drive to your Bytesized box in such a way that you can then stream using rclone cache while the files are being encrypted by rclone crypt.
If you follow these steps , at the end of the wiki you will be able to access your encrypted Google drive as well as stream from it via your Plex app.
Please note
- We don't support direct connections of your apps to the mounts that you made because connecting your apps directly to the mount can make that app highly unstable. If your apps stop responding, you will have to troubleshoot the issues yourself.
Setting up Rclone
For this wiki we will require a terminal, you can connect to your box via SSH or you can install VNC and open a terminal inside your VNC session.
To get started first create a config file. To do so type rclone config
You will see the following
#rclone config
No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q>
Press n and give it a name.
n/s/q> n
name> gdrive
Type of storage to configure.
Choose a number from below, or type in your own value (due to a very long list i will remove the list and only label the one that you have to pick)
13 / Google Drive
\ "drive"
Storage> 13
Google Application Client Id - leave blank normally.
client_id>
Google Application Client Secret - leave blank normally.
client_secret>
Scope that rclone should use when requesting access from drive.
Choose a number from below, or type in your own value
1 / Full access all files, excluding Application Data Folder.
\ "drive"
2 / Read-only access to file metadata and file contents.
\ "drive.readonly"
/ Access to files created by rclone only.
3 | These are visible in the drive website.
| File authorization is revoked when the user deauthorizes the app.
\ "drive.file"
/ Allows read and write access to the Application Data folder.
4 | This is not visible in the drive website.
\ "drive.appfolder"
/ Allows read-only access to file metadata but
5 | does not allow any access to read or download file content.
\ "drive.metadata.readonly"
scope> 1
ID of the root folder - leave blank normally. Fill in to access "Computers" folders. (see docs).
root_folder_id>
Service Account Credentials JSON file path - leave blank normally.
Needed only if you want use SA instead of interactive login.
service_account_file>
Remote config
Use auto config?
* Say Y if not sure
* Say N if you are working on a remote or headless machine or Y didn't work
y) Yes
n) No
y/n>n
If your browser doesn't open automatically go to the following link: https://accounts.google.com/o/oauth2/auth?access_type=offline&client_id=202264815644.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&response_type=code&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive&state=60234512fe651544390c517389d2e075
Log in and authorize rclone for access
Enter verification code> 4/AAApYgg421351231z4ClNa430tpgywqqzVtAUnE6nequ-bwa7YhNhlbcQoarb0
Configure this as a team drive?
y) Yes
n) No
y/n> n
I selected 11 to add a google drive account to my rClone configuration and I opened the given link in my local browser. It will redirect you to a Google login form where you can login with your Google details. Once done you should get a verification code that you paste back to rClone
--------------------
[gdrive]
type = drive
client_id =
client_secret =
scope = drive
root_folder_id =
service_account_file =
token = {"access_token":"wlawkn12312awdklawdssshhywzxr34Btfm8iv0bq7o6xD0EkmQR0dwEW8xFCULYCXpbdw-mc_eU2RnPSgHIVJS9sQwoJ_gMjV","token_type":"Bearer","refresh_token":"1/Pyk3sraAWhWZZKTJEMVIv-Qkif6iwAMn450v17CH8","expiry":"2018-03-23T16:22:27.817432122+01:00"}
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y
Current remotes:
Name Type
==== ====
gdrive drive
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q>
e/n/d/r/c/s/q> q
Setting up Rclone cache
The cache remote wraps another existing remote and stores file structure and its data for long running tasks like rclone mount
To create rclone cache first you need to go through the previous step of setting Gdrive.
Create a directory
mkdir -p ~/mnt/gdrive
Now run rclone config
#rclone config
Current remotes:
Name Type
==== ====
gdrive drive
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q>
e/n/d/r/c/s/q> n
name> gcache
Type of storage to configure.
Choose a number from below, or type in your own value (due to a very long list i will remove the list and only label the one that you have to pick)
7 / Cache a remote
\ "cache"
Storage> 7
Remote to cache.
Normally should contain a ':' and a path, eg "myremote:path/to/dir",
"myremote:bucket" or maybe "myremote:" (not recommended).
remote>gdrive:/gdrive (my folder for this wiki on google drive is called gdrive)
Optional: The URL of the Plex server
plex_url> http://eros.bysh.me:1234
Optional: The username of the Plex user
plex_username> Alchemist
Optional: The password of the Plex user
y) Yes type in my own password
g) Generate random password
n) No leave this optional password blank
y/g/n> y
Enter the password:
password:
Confirm the password:
password:
The size of a chunk. Lower value good for slow connections but can affect seamless reading.
Default: 5M
Choose a number from below, or type in your own value
1 / 1MB
\ "1m"
2 / 5 MB
\ "5M"
3 / 10 MB
\ "10M"
chunk_size> 2
How much time should object info (file size, file hashes etc) be stored in cache. Use a very high value if you don't plan on changing the source FS from outside the cache.
Accepted units are: "s", "m", "h".
Default: 6h
Choose a number from below, or type in your own value
1 / 1 hour
\ "1h"
2 / 24 hours
\ "24h"
3 / 48 hours
\ "48h"
info_age> 2
The maximum size of stored chunks. When the storage grows beyond this size, the oldest chunks will be deleted.
Default: 10G
Choose a number from below, or type in your own value
1 / 500 MB
\ "500M"
2 / 1 GB
\ "1G"
3 / 10 GB
\ "10G"
chunk_total_size> 3
Remote config
--------------------
[gcache]
type = cache
remote = gdrive:/gdrive
plex_url = http://eros.bysh.me:1234/
plex_username = Alchemist
plex_password = *** ENCRYPTED ***
chunk_size = 5M
info_age = 24h
chunk_total_size = 10G
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y
Current remotes:
Name Type
==== ====
gcache cache
gdrive drive
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q
If you don't want encryption you can mount you gcache by using the bellow command:
rclone mount gcache: ~/mnt/gdrive --allow-other --cache-db-purge --allow-non-empty --buffer-size 32M --use-mmap --dir-cache-time 72h --drive-chunk-size 16M --timeout 1h --vfs-cache-mode minimal --vfs-read-chunk-size 128M --vfs-read-chunk-size-limit 1G &
Setting up Rclone crypt
The crypt remote encrypts and decrypts another remote
To get started first create a config file by typing rclone config.
#rclone config
Current remotes:
Name Type
==== ====
gcache cache
gdrive drive
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q>n
name> gcrypt
Type of storage to configure.
Choose a number from below, or type in your own value (due to a very long list i will remove the list and only label the one that you have to pick)
10 / Encrypt/Decrypt a remote
\ "crypt"
Storage> 10
Remote to encrypt/decrypt.
Normally should contain a ':' and a path, eg "myremote:path/to/dir",
"myremote:bucket" or maybe "myremote:" (not recommended).
remote> gcache:/crypt
How to encrypt the filenames.
Choose a number from below, or type in your own value
1 / Don't encrypt the file names. Adds a ".bin" extension only.
\ "off"
2 / Encrypt the filenames see the docs for the details.
\ "standard"
3 / Very simple filename obfuscation.
\ "obfuscate"
filename_encryption> 2
Option to either encrypt directory names or leave them intact.
Choose a number from below, or type in your own value
1 / Encrypt directory names.
\ "true"
2 / Don't encrypt directory names, leave them intact.
\ "false"
directory_name_encryption>1
Password or pass phrase for encryption.
y) Yes type in my own password
g) Generate random password
y/g> y
Enter the password:
password:
Confirm the password:
password:
Password or pass phrase for salt. Optional but recommended.
Should be different to the previous password.
y) Yes type in my own password
g) Generate random password
n) No leave this optional password blank
y/g/n> g
Password strength in bits.
64 is just about memorable
128 is secure
1024 is the maximum
Bits> 128
Your password is: 12351212323
Use this password?
y) Yes
n) No
y/n> y
Remote config
--------------------
[gcrypt]
type = crypt
remote = gcache:/crypt
filename_encryption = standard
directory_name_encryption = true
password = *** ENCRYPTED ***
password2 = *** ENCRYPTED ***
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y
Current remotes:
Name Type
==== ====
gcache cache
gcrypt crypt
gdrive drive
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q
If you followed all the steps from this wiki you should now be able to access your gdrive with encrypted rclone mount command from below :
rclone mount gcrypt: ~/mnt/gdrive --allow-other --allow-non-empty --cache-db-purge --buffer-size 32M --use-mmap --dir-cache-time 72h --drive-chunk-size 16M --timeout 1h --vfs-cache-mode minimal --vfs-read-chunk-size 128M --vfs-read-chunk-size-limit 1G &
Connecting your Plex library to your mount.
Restart your Plex, once restarted go to your Plex and add a new Library. Name it whatever you want. In my example i will named it Gmovies.
Pick an option Add folders and in path there you need to type the next path:
/data/mnt/gdrive
After you have set the path , press Save changes and then run the scan of the library and you're ready to go.
HINT:
You can always manually unmount your rclone mount by using the command: fusermount -u ~/mnt/gdrive
Suggested stable startup/shutdown script
First create/open a new file called rclone by typing:
nano ~/.startup/rclone
Once created/opened , in your rclone file copy/paste the code below:
#!/bin/bash
echo "Starting Rclone"
/sbin/start-stop-daemon -S -u $USER -d $HOME --exec /usr/local/bin/rclone mount gcrypt: ~/mnt/gdrive -- --allow-other --allow-non-empty --cache-db-purge --buffer-size 32M --use-mmap --dir-cache-time 72h --drive-chunk-size 32M --timeout 1h --vfs-cache-mode minimal --vfs-read-chunk-size 128M --vfs-read-chunk-size-limit 1G &
Save the file by using the keys ctrl + o from your keyboard and then close the file by using ctrl + x.
Give the file executable permissions by typing:
chmod +x ~/.startup/rclone
Shutdown script
create/open a new file called rclone by typing:
nano ~/.shutdown/rclone
Once created/opened , in your rclone file copy/paste the code below:
#!/bin/bash
echo "Stopping rclone"
fusermount -u ~/mnt/gdrive
Save the file by using the keys ctrl + o from your keyboard and then close the file by using ctrl + x.
Give the file executable permissions by typing:
chmod +x ~/.shutdown/rclone
Using the commands
Once you have setup the startup/shutdown scripts you just need to type ~/.startup/rclone if you want to start Rclone, or~/.shutdown/rclone if you want to stop Rclone.
| Last Author | Contributors | Versions | Last update |
|---|---|---|---|
| Alchemist | None | 31 | Tue, 09 Mar 2021 15:13:55 +0100 |