Clarifying our stance on remote storage mounts (rclone/plexdrive) - Bytesized Hosting

Hi guys,

tl;dr: No ticket support for Plexdrive and Rclone issues, new button available on your boxes dashboard named "Restart box" to help you with stuck mounts.

We have seen a huge increase in members using offsite storage hosting (Google Drive and friends) in combination with our products in the last few months. In theory this is fine, however this has in turn increased the number of support tickets we are receiving to a degree where we cannot provide sufficient support. Part of that is due to the misusing of the applications to connect to these services (mainly Rclone and Plexdrive) and on the other hand a total misunderstanding of how these applications function and what you can and cannot do with them. I wanted to take some time to run through these applications, what they do, what problems arise from using them and how we are going to support them moving forward. This might get a bit technical at times, feel free to skip the technical bits if you are not interested in those.

Let's start at the lowest level, FUSE. The applications used to connect remote storage providers like Google Drive, they all use a linux system called FUSE: Filesystem in Userspace. Fuse is a translation layer in linux that can translate normal linux filesystem commands (ls for instance to list files) to any type of external call and return a valid response. Using FUSE developers can create filesystem based on web services like Google Drive. This is basically what Rclone and Plexdrive do. They take your command to list, copy or view a file from the linux filesystem, then they perform an API call to Google to return the information, the user. Now the gotcha with this is that it's an actual filesystem and file systems can break, especially when it's actually communicating with external services behind the scenes. Sadly this is bound to happen sooner or later, a remote call might return a wrong response breaking the filesystem. In linux, when a filesystem breaks nasty things start happening. Any application that has access to the FUSE mount will start hanging because they are waiting on the data from the remote provider which will never be returned. The only way to fix this is by killing all applications which are accessing the FUSE mount and then forcefully unmounting (disconnecting) the FUSE mount itself.

The other problem with FUSE mount is the performance. The performance from a FUSE mount is worse then locally attached storage in a multitude of ways.

• Increased latency, as the server needs to communicate with a seperate server which is actively storing the files.
• Overhead, for every filesystem call a translation needs to happen to a remote storage which in turn needs to hand over the requested bytes.
• Rate limits, on most services that prevent you from calling it too many times.
• Rate limits, that limit the amount of simultaneous access you have to your files.

Rclone cache & Plexdrive were created to solve part of the rate limit issues by ensuring minimal possible requests were made to the external services for data. This in turn helps in reducing the amount of requests being made but all other limitations still stand.

I want to list some things you shouldn’t expect your Rclone / Plexdrive mounts to handle in most cases:

• You should never download torrents or usenet data on your Rclone mounts. The amount of input/output operations this will cause will break your FUSE mount and/or hit your rate limits.
• You should NEVER EVER recheck torrent data; see point 1.
• Handle more then 1 or 2 simultaneous streams. It might do more in some cases but if you don't expect more you won't be disappointed.

From this moment on we are no longer accepting tickets that deal with FUSE mount performance. These mounts are completely out of our control, we can't make them faster, we can't make them more stable and therefore we won't engage in these tickets any longer. We provide you the tools to get up and running but after that it's up to you.

To help with ‘stuck’ applications on a non functional FUSE filesystem we have introduced a new button on your box’s dashboard called ‘Restart Box’, this button will attempt to unmount all (broken) FUSE mounts and stop/start all applications. This new button will not know how to restart your FUSE mount, so you will need to rerun these commands yourself.