使用Zstandard(zstd)压缩优化备份脚本，实现低负载备份网站数据

Zstandard 简介

Zstandard 也叫做 zstd，是一种快速的无损压缩算法，是针对 zlib 级别的实时压缩方案，以及更好的压缩比。它由一个非常快的熵阶段，由 Huff0 和 FSE 库提供。这个项目是作为开源的 BSD 许可收费的库，以及一个生成和解码 .zst 格式。Facebook 正在使用其快速无损压缩算法库，改进了 Facebook 的存储/备份的要求。

安装方法

升级 tar

安装前需要确认 tar 版本为 1.30 以上，验证版本号，如果低于 1.30 需要升级版本，1.30 以上可以忽略；

tar --version

下载 tar 命令的 latest 版源码压缩包：

wget https://ftp.gnu.org/gnu/tar/tar-latest.tar.gz

解压

tar xvzf tar-latest.tar.gz

编译代码

cd tar-1.3.4
FORCE_UNSAFE_CONFIGURE=1
export FORCE_UNSAFE_CONFIGURE
./configure && make

复制到运行目录

cp src/tar /bin/tar

安装 Zstandard

可以使用 yum 或者 apt 安装

#Ubuntu/Debian
apt install zstd
#Centos
yum install zstd

但这里博主推荐使用编译安装，也就两行代码

git clone https://github.com/facebook/zstd.git
cd zstd && make && sudo make install

复制到运行目录

cp /usr/local/bin/zstd /usr/bin/

具体使用方法这里就不介绍了，请自行搜索。

备份脚本

这里博主使用的是军哥 LNMP 软件包里的备份脚本修改，给出 ftp 版和 lftp 版，自行选择

lftp 版

#!/usr/bin/env bash

#Funciont: Backup website and mysql database
#Author: licess
#Website: https://lnmp.org

#IMPORTANT!!!Please Setting the following Values!

Backup_Home="/home/backup/"
MySQL_Dump="/usr/local/mysql/bin/mysqldump"
######~Set Directory you want to backup~######
Backup_Dir=("/home/wwwroot/vpser.net" "/home/wwwroot/lnmp.org")

######~Set MySQL Database you want to backup~######
Backup_Database=("lnmp" "vpser")

######~Set MySQL UserName and password~######
MYSQL_UserName='root'
MYSQL_PassWord='yourrootpassword'

######~Enable Ftp Backup~######
Enable_FTP=0
# 0: enable; 1: disable
######~Set FTP Information~######
FTP_Host='1.2.3.4'
FTP_Username='vpser.net'
FTP_Password='yourftppassword'
FTP_Dir="backup"

#Values Setting END!

TodayWWWBackup=www-*-$(date +"%Y%m%d").tar.zst
TodayDBBackup=db-*-$(date +"%Y%m%d").sql.zst
OldWWWBackup=www-*-$(date -d -30day +"%Y%m%d").tar.zst
OldDBBackup=db-*-$(date -d -30day +"%Y%m%d").sql.zst

Backup_Dir()
{
    Backup_Path=$1
    Dir_Name=`echo ${Backup_Path##*/}`
    Pre_Dir=`echo ${Backup_Path}|sed 's/'${Dir_Name}'//g'`
    tar '-I zstd -9 -T0' -cf ${Backup_Home}www-${Dir_Name}-$(date +"%Y%m%d").tar.zst -C ${Pre_Dir} ${Dir_Name}
}
Backup_Sql()
{
    ${MySQL_Dump} -u$MYSQL_UserName -p$MYSQL_PassWord $1 | zstd -9 -T0 > ${Backup_Home}db-$1-$(date +"%Y%m%d").sql.zst
}

if [ ! -f ${MySQL_Dump} ]; then  
    echo "mysqldump command not found.please check your setting."
    exit 1
fi

if [ ! -d ${Backup_Home} ]; then  
    mkdir -p ${Backup_Home}
fi

if [ ${Enable_FTP} = 0 ]; then
    type lftp >/dev/null 2>&1 || { echo >&2 "lftp command not found. Install: centos:yum install lftp,debian/ubuntu:apt-get install lftp."; }
fi

echo "Backup website files..."
for dd in ${Backup_Dir[@]};do
    Backup_Dir ${dd}
done

echo "Backup Databases..."
for db in ${Backup_Database[@]};do
    Backup_Sql ${db}
done

echo "Delete old backup files..."
rm -f ${Backup_Home}${OldWWWBackup}
rm -f ${Backup_Home}${OldDBBackup}

if [ ${Enable_FTP} = 0 ]; then
    echo "Uploading backup files to ftp..."
    cd ${Backup_Home}
    lftp ${FTP_Host} -u ${FTP_Username},${FTP_Password} << EOF
cd ${FTP_Dir}
mrm ${OldWWWBackup}
mrm ${OldDBBackup}
mput ${TodayWWWBackup}
mput ${TodayDBBackup}
bye
EOF

echo "complete."
fi

ftp 版

#!/usr/bin/env bash

#Funciont: Backup website and mysql database
#Author: licess
#Website: https://lnmp.org

#IMPORTANT!!!Please Setting the following Values!

Backup_Home="/home/backup/"
MySQL_Dump="/usr/local/mysql/bin/mysqldump"
######~Set Directory you want to backup~######
Backup_Dir=("/home/wwwroot/vpser.net" "/home/wwwroot/lnmp.org")

######~Set MySQL Database you want to backup~######
Backup_Database=("lnmp" "vpser")

######~Set MySQL UserName and password~######
MYSQL_UserName='root'
MYSQL_PassWord='yourrootpassword'

######~Enable Ftp Backup~######
Enable_FTP=0
# 0: enable; 1: disable
######~Set FTP Information~######
FTP_Host='1.2.3.4'
FTP_Username='vpser.net'
FTP_Password='yourftppassword'
FTP_Dir="backup"

#Values Setting END!

TodayWWWBackup=www-*-$(date +"%Y%m%d").tar.zst
TodayDBBackup=db-*-$(date +"%Y%m%d").sql.zst
OldWWWBackup=www-*-$(date -d -30day +"%Y%m%d").tar.zst
OldDBBackup=db-*-$(date -d -30day +"%Y%m%d").sql.zst

Backup_Dir()
{
    Backup_Path=$1
    Dir_Name=`echo ${Backup_Path##*/}`
    Pre_Dir=`echo ${Backup_Path}|sed 's/'${Dir_Name}'//g'`
    tar '-I zstd -9 -T0' -cf ${Backup_Home}www-${Dir_Name}-$(date +"%Y%m%d").tar.zst -C ${Pre_Dir} ${Dir_Name}
}
Backup_Sql()
{
    ${MySQL_Dump} -u$MYSQL_UserName -p$MYSQL_PassWord $1 | zstd -9 -T0 > ${Backup_Home}db-$1-$(date +"%Y%m%d").sql.zst
}

if [ ! -f ${MySQL_Dump} ]; then  
    echo "mysqldump command not found.please check your setting."
    exit 1
fi

if [ ! -d ${Backup_Home} ]; then  
    mkdir -p ${Backup_Home}
fi

if [ ${Enable_FTP} = 0 ]; then
    type ftp >/dev/null 2>&1 || { echo >&2 "ftp command not found. Install: centos:yum install ftp,debian/ubuntu:apt-get install lftp."; }
fi

echo "Backup website files..."
for dd in ${Backup_Dir[@]};do
    Backup_Dir ${dd}
done

echo "Backup Databases..."
for db in ${Backup_Database[@]};do
    Backup_Sql ${db}
done

echo "Delete old backup files..."
rm -f ${Backup_Home}${OldWWWBackup}
rm -f ${Backup_Home}${OldDBBackup}

if [ ${Enable_FTP} = 0 ]; then
    echo "Uploading backup files to ftp..."
    cd ${Backup_Home}
ftp -ivn ${FTP_Host} << END
user ${FTP_Username} ${FTP_Password}
type binary
cd ${FTP_Dir}
delete ${OldWWWBackup}
delete ${OldWWWBackup}
put ${TodayWWWBackup}
put ${TodayDBBackup}
bye
END

echo "complete."
fi

也可以直接使用 SFTP 协议备份到另一台服务器

if [ ${Enable_FTP} = 0 ]; then
    echo "Uploading backup files to ftp..."
    cd ${Backup_Home}
    lftp -u ${FTP_Username},${FTP_Password} sftp://${FTP_Host} << EOF
cd ${FTP_Dir}
mrm ${OldWWWBackup}
mrm ${OldDBBackup}
mput ${TodayWWWBackup}
mput ${TodayDBBackup}
bye
EOF

echo "complete."
fi

如果换了默认端口的话在 ${FTP_Host} 里，把端口加上

FTP_Host='1.2.3.4：666'

WIN 解压缩

备份后缀为 .zst 在 WIN 系统 RAR 和 7ZIP 都是无法解压的，可以安装 7ZIP 的分支，也是免费开源，点这里官方下载

结语

使用军哥的 LNMP 网站备份脚本在备份的时候 CPU 很吃力，往往都飙到 70%以上，服务器负载持续飙升的罪魁祸首是备份时的压缩算法 gzip。在改用 Zstandard 压缩，测试后 CPU 仅在 8%左右，大大减少的服务器的负担。

最近在学习这方面的资料，需要对整个系统进行异地备份，然后找到了这篇文章，感谢作者的付出。本文原文地址：https://www.yunloc.com/2438.html