Ever wanted your remote storage to perform like a local NVMe drive? This guide will show you several approaches to optimize remote storage performance on Debian/RHEL-based systems, with benchmark-proven results.

Benchmark Results
Method 1: Optimized Rclone Caching
Method 2: NFS with Local Caching
Method 3: Wireguard + Advanced I/O
Method 4: Hybrid Local/Remote Approach
Troubleshooting
Security Considerations

Benchmark Results

Here's what you can achieve with these optimization techniques:

Traditional Remote Mounting (No Optimization)

dd if=/dev/zero of=./testfile bs=1M count=100
104857600 bytes (105 MB, 100 MiB) copied, 29.1344 s, 3.6 MB/s

With Advanced Optimization Techniques

dd if=/dev/zero of=./testfile bs=1M count=100
104857600 bytes (105 MB, 100 MiB) copied, 0.144608 s, 725 MB/s

Method 1: Optimized Rclone Caching

This method uses rclone with fine-tuned parameters for maximum performance.

Prerequisites

Remote storage with SFTP access
10GB+ free local disk space for caching
Debian/Ubuntu: apt install rclone fuse
RHEL/CentOS/Fedora: dnf install epel-release && dnf install rclone fuse

Setup

Create necessary directories

mkdir -p /mnt/storage /mnt/.cache

Configure rclone

For headless servers:

rclone config

For GUI configuration (home desktop):

rclone rcd --rc-web-gui

Create a new remote with these settings:

name = storage-sftp
storage type = sftp
host = [Storage Server IP-address]
username = [Username]
port = [SSH Port - usually 22]
password = [Password]

Mount with optimized parameters

rclone mount storage-sftp:/remote/path /mnt/storage --daemon \
    --dir-cache-time 72h --cache-dir=/mnt/.cache \
    --vfs-cache-mode full --vfs-cache-max-size 10G \
    --buffer-size 256M --vfs-read-ahead 512M \
    --vfs-read-chunk-size 128M --vfs-read-chunk-size-limit 1G \
    --transfers 4 --checkers 8 --contimeout 60s \
    --timeout 300s --low-level-retries 10

Create systemd service for automatic mounting

Create file /etc/systemd/system/storage-mount.service:

[Unit]
Description=Optimized Rclone Remote Mount
AssertPathIsDirectory=/mnt/storage
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
ExecStart=/usr/bin/rclone mount storage-sftp:/remote/path /mnt/storage \
    --dir-cache-time 72h --cache-dir=/mnt/.cache \
    --vfs-cache-mode full --vfs-cache-max-size 10G \
    --buffer-size 256M --vfs-read-ahead 512M \
    --vfs-read-chunk-size 128M --vfs-read-chunk-size-limit 1G \
    --transfers 4 --checkers 8 --contimeout 60s \
    --timeout 300s --low-level-retries 10
ExecStop=/bin/fusermount -u /mnt/storage
Restart=on-failure
RestartSec=10

[Install]
WantedBy=multi-user.target

Enable and start the service

systemctl daemon-reload
systemctl enable storage-mount
systemctl start storage-mount

Method 2: NFS with Local Caching

If your storage provider supports NFS, this method often provides better performance than SFTP.

Prerequisites

Remote storage with NFS support
Debian/Ubuntu: apt install nfs-common
RHEL/CentOS/Fedora: dnf install nfs-utils

Setup

Install required packages

# Debian/Ubuntu
apt install nfs-common bcache-tools

# RHEL/CentOS/Fedora
dnf install nfs-utils bcache-tools

Create an NVMe cache partition

Warning: This will erase data on the specified partition!

# Identify your NVMe device
lsblk

# Create a bcache device
# /dev/nvme0n1p1 is your NVMe partition for caching
# Replace X with your actual device
make-bcache -C /dev/nvme0n1p1

Mount the NFS share

mkdir -p /mnt/storage-backend
mount -t nfs storage.server:/path /mnt/storage-backend

Set up bcache with the NFS as backend

# Register the NFS mount as a bcache backend device
echo /mnt/storage-backend > /sys/fs/bcache/register

# Find the bcache device name
ls /dev/bcache*
mkdir -p /mnt/storage
mount /dev/bcache0 /mnt/storage

Create systemd service files for automatic mounting

Create file /etc/systemd/system/storage-nfs-backend.service:

[Unit]
Description=NFS Backend Mount
After=network-online.target
Wants=network-online.target

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/mount -t nfs storage.server:/path /mnt/storage-backend
ExecStop=/bin/umount /mnt/storage-backend

[Install]
WantedBy=multi-user.target

Create file /etc/systemd/system/storage-cached.service:

[Unit]
Description=Bcache NFS Mount
After=storage-nfs-backend.service
Requires=storage-nfs-backend.service

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/sh -c 'echo /mnt/storage-backend > /sys/fs/bcache/register && mount /dev/bcache0 /mnt/storage'
ExecStop=/bin/umount /mnt/storage

[Install]
WantedBy=multi-user.target

Enable and start the services

systemctl daemon-reload
systemctl enable storage-nfs-backend storage-cached
systemctl start storage-nfs-backend storage-cached

Method 3: Wireguard + Advanced I/O

Using Wireguard instead of traditional SSH reduces encryption overhead while maintaining security.

Prerequisites

Control over both the client and remote server
Debian/Ubuntu: apt install wireguard-tools
RHEL/CentOS/Fedora: dnf install wireguard-tools

Setup

Install Wireguard on both systems

# Debian/Ubuntu
apt install wireguard-tools

# RHEL/CentOS/Fedora
dnf install wireguard-tools

Generate keys on local system

wg genkey | tee local-private.key | wg pubkey > local-public.key

Generate keys on remote storage server

wg genkey | tee remote-private.key | wg pubkey > remote-public.key

Configure Wireguard on local system

Create file /etc/wireguard/wg0.conf:

[Interface]
PrivateKey = <local-private-key>
Address = 10.10.10.1/24
ListenPort = 51820

[Peer]
PublicKey = <remote-public-key>
AllowedIPs = 10.10.10.2/32
Endpoint = <remote-server-ip>:51820
PersistentKeepalive = 25

Configure Wireguard on remote storage server

Create file /etc/wireguard/wg0.conf:

[Interface]
PrivateKey = <remote-private-key>
Address = 10.10.10.2/24
ListenPort = 51820

[Peer]
PublicKey = <local-public-key>
AllowedIPs = 10.10.10.1/32

Start Wireguard on both systems

wg-quick up wg0
systemctl enable wg-quick@wg0

Optimize network parameters on both systems

# Create a file /etc/sysctl.d/99-network-performance.conf with these contents:
cat > /etc/sysctl.d/99-network-performance.conf << EOF
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.ipv4.tcp_congestion_control = bbr
net.core.netdev_max_backlog = 5000
EOF

# Apply changes
sysctl -p /etc/sysctl.d/99-network-performance.conf

Mount using NFS over Wireguard with io_uring

On the remote storage server, configure NFS:

# Debian/Ubuntu
apt install nfs-kernel-server

# RHEL/CentOS/Fedora
dnf install nfs-utils

# Export the directory
echo '/storage 10.10.10.1(rw,sync,no_subtree_check)' >> /etc/exports
exportfs -a
systemctl restart nfs-server

On the local system:

mkdir -p /mnt/storage
mount -t nfs -o vers=4.2,fsc,nocto,noatime,nodiratime 10.10.10.2:/storage /mnt/storage

# For Linux kernel 5.1+ with io_uring support
mount -t nfs -o vers=4.2,fsc,nocto,noatime,nodiratime,io_uring 10.10.10.2:/storage /mnt/storage

Create systemd service for automatic mounting

Create file /etc/systemd/system/storage-wireguard.service:

[Unit]
Description=Mount storage over Wireguard
After=network-online.target wg-quick@wg0.service
Requires=wg-quick@wg0.service

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/mount -t nfs -o vers=4.2,fsc,nocto,noatime,nodiratime 10.10.10.2:/storage /mnt/storage
ExecStop=/bin/umount /mnt/storage

[Install]
WantedBy=multi-user.target

Enable and start the service

systemctl daemon-reload
systemctl enable storage-wireguard
systemctl start storage-wireguard

Method 4: Hybrid Local/Remote Approach

This approach gives you true local NVMe performance while automatically syncing to remote storage.

Prerequisites

Debian/Ubuntu: apt install lsyncd
RHEL/CentOS/Fedora: dnf install epel-release && dnf install lsyncd

Setup

Install required packages

# Debian/Ubuntu
apt install lsyncd

# RHEL/CentOS/Fedora
dnf install epel-release
dnf install lsyncd

Create local and remote mount points

mkdir -p /mnt/local-nvme /mnt/remote-storage

Mount your local NVMe drive

# Replace with your actual NVMe partition
mount /dev/nvme0n1p1 /mnt/local-nvme

Mount your remote storage using one of the previous methods

# Example using rclone
rclone mount storage-sftp:/remote/path /mnt/remote-storage --daemon

Configure lsyncd for real-time synchronization

Create file /etc/lsyncd/lsyncd.conf.lua:

settings {
   logfile = "/var/log/lsyncd/lsyncd.log",
   statusFile = "/var/log/lsyncd/lsyncd-status.log",
   statusInterval = 10
}

sync {
   default.rsync,
   source = "/mnt/local-nvme/",
   target = "/mnt/remote-storage/",
   delay = 5,
   rsync = {
      archive = true,
      compress = true,
      bwlimit = 10000  -- 10MB/s bandwidth limit
   }
}

Create necessary log directories

mkdir -p /var/log/lsyncd

Start and enable lsyncd

systemctl start lsyncd
systemctl enable lsyncd

Create convenience symbolic links (optional)

ln -s /mnt/local-nvme /storage

Troubleshooting

Connection Issues

# Check if the mount is responsive
ls -la /mnt/storage

# Check system logs for errors
journalctl -u storage-mount -n 50

# Restart the mount service
systemctl restart storage-mount

# Force unmount if stuck
fusermount -uz /mnt/storage

Performance Degradation

# Check disk space for cache
df -h /mnt/.cache

# Clear rclone cache (if using rclone)
rm -rf /mnt/.cache/*
systemctl restart storage-mount

# Check network performance
iperf3 -c <remote-server-ip>

Monitoring Mount Health

# Monitor ongoing transfers with rclone
rclone rc core/stats

# Watch real-time I/O activity
iostat -xm 2

Security Considerations

Use key-based authentication instead of password Generate an SSH key pair:

   ssh-keygen -t ed25519 -f ~/.ssh/storage_key

For rclone, update your config to use the key:

   rclone config

Choose your remote, then:

   key_file = /root/.ssh/storage_key
   key_use_agent = false

Encrypt sensitive data If storing sensitive data, use rclone crypt:

   rclone config

Create a new remote of type "crypt" that points to your existing remote.

Restrict mount permissions

   chmod 700 /mnt/storage

Remember that while these methods create the impression of local access speeds, actual file transfers to the remote storage still happen in the background. For the best experience, ensure a stable and fast internet connection.

Source & Attribution

This article is based on original data belonging to ENGINYRING.COM blog. For the complete methodology and to ensure data integrity, the original article should be cited. The canonical source is available at: Advanced Remote Storage Optimization: Making Remote Storage Feel Like Local NVMe.

Advanced Remote Storage Optimization: Making Remote Storage Feel Like Local NVMe

Table of Contents

Benchmark Results

Method 1: Optimized Rclone Caching

Prerequisites

Setup

Method 2: NFS with Local Caching

Prerequisites

Setup

Method 3: Wireguard + Advanced I/O

Prerequisites

Setup

Method 4: Hybrid Local/Remote Approach

Prerequisites

Setup

Troubleshooting

Connection Issues

Performance Degradation

Monitoring Mount Health

Security Considerations

Source & Attribution

You May Also Like

A Guide to Tuning Kernel Parameters with sysctl in Linux

How to Install and Optimize Magento for Speed on Your Linux VPS (Debian/Ubuntu)

How to Detect Intruders in Your VPS Server: Complete Security Guide

VPS Hosting for DevOps Pipelines: Setting Up GitLab CI/CD on Your Own Server

Misc

Hosting

Management & Branding

Digitalization

Advanced Remote Storage Optimization: Making Remote Storage Feel Like Local NVMe

Table of Contents

Benchmark Results

Method 1: Optimized Rclone Caching

Prerequisites

Setup

Method 2: NFS with Local Caching

Prerequisites

Setup

Method 3: Wireguard + Advanced I/O

Prerequisites

Setup

Method 4: Hybrid Local/Remote Approach

Prerequisites

Setup

Troubleshooting

Connection Issues

Performance Degradation

Monitoring Mount Health

Security Considerations

Source & Attribution

Share This Post

You May Also Like