使用 Rclone 同步文件到 Google Drive
目前我使用 Calibre 软件来管理电子书,电子书备份在 iCloud 中。但是每次推送电子书到 Kindle 都要打开电脑,感觉不够方便。如果能在手机端就可以操作岂不是更香。
遂将目标瞄向了 calibre-web,这个类似 calibre 的 web 端。可以浏览、下载 Calibre 数据库中保存的书籍,也有 Docker 镜像提供。我有一个 x86 的虚拟机用来补全 m1 的短板,可以用来运行 calibre-web 容器。
接下来就是数据的持久化了。众所周知 iCloud 一如 Apple 的系统一样封闭,无法在 Linux 上使用。想起 Google Drive 有 15GB 的免(bai)费(piao)空间,可以用来备份电子书。记得 Rclone 可以支持 Google Drive 的同步(网络问题不在讨论范围),虽然只有单向同步但正好满足我的需求。
上面就是最终的理想设计了,这篇先进行虚拟机的安装实现本地文件到 Google Drive 的同步。
Rclone
Rclone 是一个命令行程序,用于管理云存储上的文件。它是一个开源的软件,灵感来自 rsync。支持多种操作系统,以及 x86、arm 计算架构。
Rclone 支持的云存储有 60 多种,具体列表可以参考这里。
动手
Google API client_id
Rclone 访问 Google Drive,需要通过 Google API 来完成。首先我们要为创建 Rclone 创建一个 client_id,详细步骤参考这里。
配置 Rclone
命令行中输入 rclone config
,然后按照详细步骤一步步完成配置。
由于在创建 client_id 时,应用类型选的是“桌面”应用。在配置 Rclone 时,需要在浏览器中打开 http://127.0.0.1:53682/auth
操作完成 Google Drive 的访问授权,这在虚拟机中无法完成,但可以使用代理的方式来解决。
从 release 中下载最新版的 Pipy,然后创建一个脚本文件:
#/tmp/proxy.js
pipy()
.listen(3682)
.connect('127.0.0.1:53682')
然后,就可以使用命令 ./pipy /tmp/proxy.js
启动代理。然后我们通过虚拟机的 IP 地址和端口 3682
,在浏览器中完成授权。授权后,有个回调的操作同样修改 IP 地址和端口来完成。
其他的步骤就按照文档中的指示完成操作。然后执行命令 rclone ls remote
,如果返回了 Google Drive 的文件列表,说明配置成功。
通过 Rclone 的 sync
命令可以完成同步,比如将本地 /data/calibre-web
的内容同步到 Google Drive 的 calibre
目录中。
$ rclone sync /data/calibre-web remote:calibre
自动同步
在每次修改文件内容后需要手动执行一次 sync
操作进行同步,很不方便。除了通过 crontab 在周期性的同步外,还可以通过监控文件操作来实现自动的同步。
网上就有人提供了自动同步的脚本,使用 inotifywait
来监控文件的修改来进行同步。
将脚本内容保存为 sync.sh
,修改其中的几个参数即可。记得执行脚本前,需要先安装 inotify-toole
和 libnotify-bin
。
$ sudo apt install libnotify-bin inotify-tools
#!/bin/bash
## One-way, immediate, continuous, recursive, directory synchronization
## to a remote Rclone URL. ( S3, SFTP, FTP, WebDAV, Dropbox, etc. )
## Optional desktop notifications on sync events or errors.
## Useful only for syncing a SMALL number of files (< 8192).
## (See note in `man inotifywait` under `--recursive` about raising this limit.)
## Think use-case: Synchronize Traefik TLS certificates file (acme.json)
## Think use-case: Synchronize Keepass (.kdbx) database file immediately on save.
## Think use-case: Live edit source code and push to remote server
## This is NOT a backup tool!
## It will not help you if you delete your files or if they become corrupted.
## Setup: Install `rclone` from package manager.
## Run: `rclone config` to setup the remote, including the full remote
## subdirectory path to sync to.
## MAKE SURE that the remote (sub)directory is EMPTY
## or else ALL CONTENTS WILL BE DELETED by rclone when it syncs.
## If unsure, add `--dry-run` to the RCLONE_CMD variable below,
## to simulate what would be copied/deleted.
## Copy this script any place, and make it executable: `chown +x sync.sh`
## Edit all the variables below, before running the script.
## Run: `./sync.sh systemd_setup` to create and enable systemd service.
## Run: `journalctl --user --unit rclone_sync.${RCLONE_REMOTE}` to view the logs.
## Edit the variables below, according to your own environment:
# RCLONE_SYNC_PATH: The path to COPY FROM (files are not synced TO here):
RCLONE_SYNC_PATH="/data/calibre-web"
# RCLONE_REMOTE: The rclone remote name to synchronize with.
# Identical to one of the remote names listed via `rclone listremotes`.
# Make sure to include the final `:` in the remote name, which
# indicates to sync/delete from the same (sub)directory as defined in the URL.
# (ALL CONTENTS of the remote are continuously DELETED
# and replaced with the contents from RCLONE_SYNC_PATH)
RCLONE_REMOTE="remote:calibre"
# RCLONE_CMD: The sync command and arguments:
RCLONE_CMD="rclone -v sync ${RCLONE_SYNC_PATH} ${RCLONE_REMOTE}"
# WATCH_EVENTS: The file events that inotifywait should watch for:
WATCH_EVENTS="modify,delete,create,move"
# SYNC_DELAY: Wait this many seconds after an event, before synchronizing:
SYNC_DELAY=5
# SYNC_INTERVAL: Wait this many seconds between forced synchronizations:
SYNC_INTERVAL=3600
# NOTIFY_ENABLE: Enable Desktop notifications
NOTIFY_ENABLE=true
# SYNC_SCRIPT: dynamic reference to the current script path
SYNC_SCRIPT=$(realpath $0)
notify() {
MESSAGE=$1
if test ${NOTIFY_ENABLE} = "true"; then
notify-send "rclone ${RCLONE_REMOTE}" "${MESSAGE}"
fi
}
rclone_sync() {
set -x
# Do initial sync immediately:
notify "Startup"
${RCLONE_CMD}
# Watch for file events and do continuous immediate syncing
# and regular interval syncing:
while [[ true ]] ; do
inotifywait --recursive --timeout ${SYNC_INTERVAL} -e ${WATCH_EVENTS} \
${RCLONE_SYNC_PATH} 2>/dev/null
if [ $? -eq 0 ]; then
# File change detected, sync the files after waiting a few seconds:
sleep ${SYNC_DELAY} && ${RCLONE_CMD} && \
notify "Synchronized new file changes"
elif [ $? -eq 1 ]; then
# inotify error occured
notify "inotifywait error exit code 1"
sleep 10
elif [ $? -eq 2 ]; then
# Do the sync now even though no changes were detected:
${RCLONE_CMD}
fi
done
}
systemd_setup() {
set -x
if loginctl show-user ${USER} | grep "Linger=no"; then
echo "User account does not allow systemd Linger."
echo "To enable lingering, run as root: loginctl enable-linger $USER"
echo "Then try running this command again."
exit 1
fi
mkdir -p ${HOME}/.config/systemd/user
SERVICE_FILE=${HOME}/.config/systemd/user/rclone_sync.${RCLONE_REMOTE}.service
if test -f ${SERVICE_FILE}; then
echo "Unit file already exists: ${SERVICE_FILE} - Not overwriting."
else
cat <<EOF > ${SERVICE_FILE}
[Unit]
Description=rclone_sync ${RCLONE_REMOTE}
[Service]
ExecStart=${SYNC_SCRIPT}
[Install]
WantedBy=default.target
EOF
fi
systemctl --user daemon-reload
systemctl --user enable --now rclone_sync.${RCLONE_REMOTE}
systemctl --user status rclone_sync.${RCLONE_REMOTE}
echo "You can watch the logs with this command:"
echo " journalctl --user --unit rclone_sync.${RCLONE_REMOTE}"
}
if test $# = 0; then
rclone_sync
else
CMD=$1; shift;
${CMD} $@
fi