Terminal Scripts

Rotating rclone uploads across multiple Google Shared Drives

For those days when everything downloads and you get API blocked by Google....

It happens! If you think you may have days when you need to upload more than 750GB a day, the following approach works:

Use a Shared Drive (formerly "Team Drive") rather than a normal Google Drive. This gives other accounts you share it with full access without sharing your upload or API limits.
Create multiple normal Google accounts (no paid accounts needed).
Share your Shared Drive with these accounts. It can take a couple of minutes to appear in the other Drives.
Using rclone config, create multiple remotes, one per account. Make sure you set them up as Shared Drives when the option appears.

Once the above is done, use the following script to upload your downloaded files. A mount will not work!

I have this set up to run hourly via crontab — feel free to use whatever schedule you'd like. Bear in mind that having two instances running at the same time could cause issues.

My drives are named:

GDrive-Team
GDrive-Team1
GDrive-Team2
GDrive-Team3

Feel free to name them whatever you want, but edit the script accordingly. You can have as many or as few as you'd like. Also remember to edit the source and destination folders to match your own folder structure.

Note: The last sync block is slightly different — it handles a secondary video download script that does not like files disappearing mid-transfer. It checks for .part files and if any are present, skips transferring that folder until next run. You probably don't need this.

#!/bin/sh

lastsync=$(tail -1 lastsync.txt | head -1)

if [ $lastsync = "Team" ]; then
    teamnumber=""
    echo "Team1" > lastsync.txt

elif [ $lastsync = "Team1" ]; then
    teamnumber="1"
    echo "Team2" > lastsync.txt

elif [ $lastsync = "Team2" ]; then
    teamnumber="2"
    echo "Team3" > lastsync.txt

elif [ $lastsync = "Team3" ]; then
    teamnumber="3"
    echo "Team" > lastsync.txt
fi

echo "Using Team Account $lastsync to save to Drive"

driveToUse="GDrive-Team$teamnumber"

# Edit the path below to match your own folder structure
SRCpath="/home/username/ToMove"

echo $driveToUse

seriesSRC="$SRCpath/Unorganised Series"
filmsSRC="$SRCpath/Unorganised Films"
musicSRC="$SRCpath/Unorganised Music"
appsSRC="$SRCpath/Downloaded Apps"
booksSRC="$SRCpath/E-Books"
dlvideoSRC="$SRCpath/Videos-To-Move"

seriesDST="$driveToUse:Unorganised Series"
filmsDST="$driveToUse:Unorganised Films"
musicDST="$driveToUse:Unorganised Music"
appsDST="$driveToUse:Downloaded Apps"
booksDST="$driveToUse:Unorganised Books"
dlvideoDST="$driveToUse:Plex Cloud Videos"

rclone --exclude /hold --retries 50 move -v "$seriesSRC" "$seriesDST"
rclone --exclude /hold --retries 50 move -v "$filmsSRC"  "$filmsDST"
rclone --exclude /hold --retries 50 move -v "$musicSRC"  "$musicDST"
rclone --exclude /hold --retries 50 move -v "$appsSRC"   "$appsDST"
rclone --exclude /hold --retries 50 move -v "$booksSRC"  "$booksDST"

# Skip video folder if .part files are present (incomplete downloads)
count=`ls "$dlvideoSRC"/*.part 2>/dev/null | wc -l`
echo ".Part files existing: $count"
if [ $count -lt 1 ]; then
    echo "Video Sync Running"
    rclone --exclude /hold --retries 50 move -v "$dlvideoSRC" "$dlvideoDST"
fi