使用 Rclone 同步文件到 Google Drive

使用 Rclone 同步文件到 Google Drive

目前我使用 Calibre 软件来管理电子书,电子书备份在 iCloud 中。但是每次推送电子书到 Kindle 都要打开电脑,感觉不够方便。如果能在手机端就可以操作岂不是更香。

遂将目标瞄向了 calibre-web,这个类似 calibre 的 web 端。可以浏览、下载 Calibre 数据库中保存的书籍,也有 Docker 镜像提供。我有一个 x86 的虚拟机用来补全 m1 的短板,可以用来运行 calibre-web 容器。

接下来就是数据的持久化了。众所周知 iCloud 一如 Apple 的系统一样封闭,无法在 Linux 上使用。想起 Google Drive 有 15GB 的免(bai)费(piao)空间,可以用来备份电子书。记得 Rclone 可以支持 Google Drive 的同步(网络问题不在讨论范围),虽然只有单向同步但正好满足我的需求。

rclone

上面就是最终的理想设计了,这篇先进行虚拟机的安装实现本地文件到 Google Drive 的同步。

Rclone

Rclone 是一个命令行程序,用于管理云存储上的文件。它是一个开源的软件,灵感来自 rsync。支持多种操作系统,以及 x86、arm 计算架构。

Rclone 支持的云存储有 60 多种,具体列表可以参考这里

动手

Google API client_id

Rclone 访问 Google Drive,需要通过 Google API 来完成。首先我们要为创建 Rclone 创建一个 client_id,详细步骤参考这里

配置 Rclone

命令行中输入 rclone config,然后按照详细步骤一步步完成配置。

由于在创建 client_id 时,应用类型选的是“桌面”应用。在配置 Rclone 时,需要在浏览器中打开 http://127.0.0.1:53682/auth 操作完成 Google Drive 的访问授权,这在虚拟机中无法完成,但可以使用代理的方式来解决。

从 release 中下载最新版的 Pipy,然后创建一个脚本文件:

#/tmp/proxy.js
pipy()

.listen(3682)
  .connect('127.0.0.1:53682')

然后,就可以使用命令 ./pipy /tmp/proxy.js 启动代理。然后我们通过虚拟机的 IP 地址和端口 3682,在浏览器中完成授权。授权后,有个回调的操作同样修改 IP 地址和端口来完成。

其他的步骤就按照文档中的指示完成操作。然后执行命令 rclone ls remote,如果返回了 Google Drive 的文件列表,说明配置成功。

通过 Rclone 的 sync 命令可以完成同步,比如将本地 /data/calibre-web 的内容同步到 Google Drive 的 calibre 目录中。

$ rclone sync /data/calibre-web remote:calibre

自动同步

在每次修改文件内容后需要手动执行一次 sync 操作进行同步,很不方便。除了通过 crontab 在周期性的同步外,还可以通过监控文件操作来实现自动的同步。

网上就有人提供了自动同步的脚本,使用 inotifywait 来监控文件的修改来进行同步。

将脚本内容保存为 sync.sh,修改其中的几个参数即可。记得执行脚本前,需要先安装 inotify-toolelibnotify-bin

$ sudo apt install libnotify-bin inotify-tools
#!/bin/bash
## One-way, immediate, continuous, recursive, directory synchronization
##  to a remote Rclone URL. ( S3, SFTP, FTP, WebDAV, Dropbox, etc. )
## Optional desktop notifications on sync events or errors.
## Useful only for syncing a SMALL number of files (< 8192).
##  (See note in `man inotifywait` under `--recursive` about raising this limit.)
## Think use-case: Synchronize Traefik TLS certificates file (acme.json)
## Think use-case: Synchronize Keepass (.kdbx) database file immediately on save.
## Think use-case: Live edit source code and push to remote server

## This is NOT a backup tool! 
## It will not help you if you delete your files or if they become corrupted.

## Setup: Install `rclone` from package manager.
## Run: `rclone config` to setup the remote, including the full remote
##   subdirectory path to sync to.
## MAKE SURE that the remote (sub)directory is EMPTY
##   or else ALL CONTENTS WILL BE DELETED by rclone when it syncs.
## If unsure, add `--dry-run` to the RCLONE_CMD variable below, 
##   to simulate what would be copied/deleted. 
## Copy this script any place, and make it executable: `chown +x sync.sh`
## Edit all the variables below, before running the script.
## Run: `./sync.sh systemd_setup` to create and enable systemd service.
## Run: `journalctl --user --unit rclone_sync.${RCLONE_REMOTE}` to view the logs.

## Edit the variables below, according to your own environment:

# RCLONE_SYNC_PATH: The path to COPY FROM (files are not synced TO here):
RCLONE_SYNC_PATH="/data/calibre-web"

# RCLONE_REMOTE: The rclone remote name to synchronize with.
# Identical to one of the remote names listed via `rclone listremotes`.
# Make sure to include the final `:` in the remote name, which
#   indicates to sync/delete from the same (sub)directory as defined in the URL.
# (ALL CONTENTS of the remote are continuously DELETED
#  and replaced with the contents from RCLONE_SYNC_PATH)
RCLONE_REMOTE="remote:calibre"

# RCLONE_CMD: The sync command and arguments:
RCLONE_CMD="rclone -v sync ${RCLONE_SYNC_PATH} ${RCLONE_REMOTE}"

# WATCH_EVENTS: The file events that inotifywait should watch for:
WATCH_EVENTS="modify,delete,create,move"

# SYNC_DELAY: Wait this many seconds after an event, before synchronizing:
SYNC_DELAY=5

# SYNC_INTERVAL: Wait this many seconds between forced synchronizations:
SYNC_INTERVAL=3600

# NOTIFY_ENABLE: Enable Desktop notifications
NOTIFY_ENABLE=true

# SYNC_SCRIPT: dynamic reference to the current script path
SYNC_SCRIPT=$(realpath $0)

notify() {
    MESSAGE=$1
    if test ${NOTIFY_ENABLE} = "true"; then
        notify-send "rclone ${RCLONE_REMOTE}" "${MESSAGE}"
    fi
}

rclone_sync() {
    set -x
    # Do initial sync immediately:
    notify "Startup"
    ${RCLONE_CMD}
    # Watch for file events and do continuous immediate syncing
    # and regular interval syncing:
    while [[ true ]] ; do
	inotifywait --recursive --timeout ${SYNC_INTERVAL} -e ${WATCH_EVENTS} \
		    ${RCLONE_SYNC_PATH} 2>/dev/null
	if [ $? -eq 0 ]; then
	    # File change detected, sync the files after waiting a few seconds:
	    sleep ${SYNC_DELAY} && ${RCLONE_CMD} && \
		notify "Synchronized new file changes"
	elif [ $? -eq 1 ]; then
	    # inotify error occured
	    notify "inotifywait error exit code 1"
        sleep 10
	elif [ $? -eq 2 ]; then
	    # Do the sync now even though no changes were detected:
	    ${RCLONE_CMD}
	fi
    done
}

systemd_setup() {
    set -x
    if loginctl show-user ${USER} | grep "Linger=no"; then
	    echo "User account does not allow systemd Linger."
	    echo "To enable lingering, run as root: loginctl enable-linger $USER"
	    echo "Then try running this command again."
	    exit 1
    fi
    mkdir -p ${HOME}/.config/systemd/user
    SERVICE_FILE=${HOME}/.config/systemd/user/rclone_sync.${RCLONE_REMOTE}.service
    if test -f ${SERVICE_FILE}; then
	    echo "Unit file already exists: ${SERVICE_FILE} - Not overwriting."
    else
	    cat <<EOF > ${SERVICE_FILE}
[Unit]
Description=rclone_sync ${RCLONE_REMOTE}

[Service]
ExecStart=${SYNC_SCRIPT}

[Install]
WantedBy=default.target
EOF
    fi
    systemctl --user daemon-reload
    systemctl --user enable --now rclone_sync.${RCLONE_REMOTE}
    systemctl --user status rclone_sync.${RCLONE_REMOTE}
    echo "You can watch the logs with this command:"
    echo "   journalctl --user --unit rclone_sync.${RCLONE_REMOTE}"
}

if test $# = 0; then
    rclone_sync
else
    CMD=$1; shift;
    ${CMD} $@
fi
comments powered by Disqus