Install and setup Backup Manager on Debian
Backup Manager is a tool easing data backup. It offer many options and backup methods. It is perfectly adapted to the backup of a single dedicated server.
This howto is tested on:
- Debian 5.0 Lenny
- Debian 6.0 Squeeze
Installation
Install backup-manager with its default configuration:
DEBIAN_FRONTEND='noninteractive' command apt-get install backup-manager bzip2
Allow the "backup" group to view, but not edit, archives :
command sed -i -e 's/[#]*\(.*BM_REPOSITORY_GROUP=\).*$/\1"backup"/' \
-e 's/[#]*\(.*BM_REPOSITORY_CHMOD=\).*$/\1"750"/' \
-e 's/[#]*\(.*BM_ARCHIVE_CHMOD=\).*$/\1"640"/' \
'/etc/backup-manager.conf'
Add the log level option to prevent warnings :
if [ -z "$(command grep 'BM_LOGGER_LEVEL' '/etc/backup-manager.conf')" ]; then command echo ' export BM_LOGGER_LEVEL="warning"' \ >> '/etc/backup-manager.conf' fi
Setup a default port for SSH uploads, to prevent warnings on Debian Lenny :
command sed -i \
-e 's/[#]*\(.*BM_UPLOAD_SSH_PORT=\).*$/\1"22"/' \
'/etc/backup-manager.conf'
Set the archives file type to tar.bz2 :
command sed -i \
-e 's/[#]*\(.*BM_TARBALL_FILETYPE=\).*$/\1"tar.bz2"/' \
'/etc/backup-manager.conf'
Enable incremental backup :
command sed -i \
-e 's/[#]*\(.*BM_ARCHIVE_METHOD=\).*$/\1"tarball-incremental"/' \
'/etc/backup-manager.conf'
By default, "master" backups are created on mondays.
Use the BM_TARBALL_TARGETS option in place of BM_TARBALL_DIRECTORIES. It ease the management of backup targets :
command sed -i \
-e 's/^# \(declare.*BM_TARBALL_TARGETS.*\)$/\1/' \ -e 's/^# \(BM_TARBALL_TARGETS\[.*\)$/\1/' \ -e 's/^# \(export.*BM_TARBALL_TARGETS.*\)$/\1/' \
-e 's/^\(export BM_TARBALL_DIRECTORIES.*\)$/# \1/' \
'/etc/backup-manager.conf'
Note : if the "/var/archives" folder contains other systems backups, enable the purge of this other systems old backups :
command sed -i \
-e 's/[#]*\(.*BM_ARCHIVE_STRICTPURGE=\).*$/\1"false"/' \
'/etc/backup-manager.conf'
Backup targets management
Install the backup target management tool :
command wget 'https://raw.github.com/biapy/howto.biapy.com/master/backup-manager/backup-manager-tools' \
--quiet --no-check-certificate --output-document='/usr/bin/backup-manager-tools'
command chmod +x '/usr/bin/backup-manager-tools'
This tool usage is :
- Add a target to backup :
command backup-manager-tools add '/mon/dossier'
- Remove a target from backup :
command backup-manager-tools remove '/mon/dossier'
- Clean-up deleted backup targets :
command backup-manager-tools cleanup
By default, Backup Manager is setup to backup /etc and /home folders. Add to this /root and /var/backups :
command backup-manager-tools add '/root' command backup-manager-tools add '/var/backups'
Optimizations
Backup scheduling
By default, the backup is done with others daily tasks (cron.daily) at 6:25 AM. Change the cron.daily schedule in the file "/etc/crontab" (the bold values are the minute and hour settings to customize):
# m h dom mon dow user command
25 6 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
Reload the cron configuration:
/etc/init.d/cron reload
Backup process priority
If the system responsiveness is important, set to the lowest level the backup process priority. The backup will take more time, but will not interfere with the system other works:
command sed -i \
-e 's/[#]*\(.*BM_ARCHIVE_NICE_LEVEL=\).*$/\1"19"/' \
'/etc/backup-manager.conf'
Archives compression
The type of created archives affects the system load during the backup. If the volume of data is important, and the disk space used by backups is not a critical data, change the file type for tar archives (no compression is applied to backups):
command sed -i \
-e 's/[#]*\(.*BM_TARBALL_FILETYPE=\).*$/\1"tar"/' \
'/etc/backup-manager.conf'
Backups export
A hardware break is always possible. To prevent data loss, setup a export of backed-up data to another location. Backup Manager can copy its backups to other hosts.
SCP export
This method make use of a remote host with SSH access. This method need to be able to connect to a distant account with a authentication SSL key (see ssh-keygen and ssh-copy-id). Provide the SSH server connection parameters (edit the bold values) :
SSH_USER="backup" SSH_PRIVATE_KEY="/root/backup-rsa-key" SSH_HOST="server.domain.com" SSH_PATH="/var/archives/"
The SSH private key can be created with ssh-keygen.
Update the software configuration :
command sed -i -e "s|[#]*\(.*BM_UPLOAD_SSH_USER=\).*$|\1\"${SSH_USER}\"|" \
-e "s|[#]*\(.*BM_UPLOAD_SSH_KEY=\).*$|\1\"${SSH_PRIVATE_KEY}\"|" \
-e "s|[#]*\(.*BM_UPLOAD_SSH_HOSTS=\).*$|\1\"${SSH_HOST}\"|" \
-e "s|[#]*\(.*BM_UPLOAD_SSH_DESTINATION=\).*$|\1\"${SSH_PATH}\"|" \
-e "s|[#]*\(.*BM_UPLOAD_SSH_PORT=\).*$|\1\"22\"|" \
'/etc/backup-manager.conf'
Enable the SCP export :
command sed -i -e "s/[#]*\(.*BM_UPLOAD_METHOD=\).*$/\1\"scp\"/" \
'/etc/backup-manager.conf'
The BM_UPLOAD_SSH_PORT setting is needed for this method to work on Debian 5.0 Lenny. The BM_UPLOAD_SSH_TTL setting is needed to enable the purge of old distant backups.
Rsync export
This method allow to backup heavy folders on a remote server, without the need to create a tarball. It is the recommended way to backup folders with a size over one Gigabyte.
By default, this method use the SSH server configured in SCP export.
Provide the list of folders to be backuped with Rsync:
RSYNC_DIRECTORIES="/var/archives"
Update the software configuration:
command sed -i -e "s|[#]*\(.*BM_UPLOAD_RSYNC_DIRECTORIES=\).*$|\1\"${RSYNC_DIRECTORIES}\"|" \
'/etc/backup-manager.conf'
Enable the Rsync export in addition to the current upload method:
command sed -i -e "s/[#]*\(.*BM_UPLOAD_METHOD=\"[^\"]*\)\".*$/\1 rsync\"/" \
'/etc/backup-manager.conf'
Or enable the Rsync export only:
command sed -i -e "s/[#]*\(.*BM_UPLOAD_METHOD=\).*$/\1\"rsync\"/" \
'/etc/backup-manager.conf'
Note: If the SSH server use a custom port, apply this patch for Backup Manager to use it:
command dpkg-divert --divert '/usr/share/backup-manager/upload-methods.sh.dpkg-orig' \
--rename '/usr/share/backup-manager/upload-methods.sh'
command cp -a '/usr/share/backup-manager/upload-methods.sh.dpkg-orig' '/usr/share/backup-manager/upload-methods.sh'
command sed -i -e '/ssh_option="ssh -l/i\
if [[ -n "${BM_UPLOAD_SSH_PORT}" ]]; then\
ssh_port_switch="-p ${BM_UPLOAD_SSH_PORT}";\
fi' \
-e 's/ssh_option="ssh -l/ssh_option="ssh ${ssh_port_switch} -l/' \
'/usr/share/backup-manager/upload-methods.sh'
FTP export
This method make use of a remote host with FTP access (note that OVH provide a free FTP backup space for each dedicated server). Provide the FTP access parameters (edit the bold values) :
FTP_HOST='ftp.ovh.fr' FTP_USER='some-login' FTP_PASSWORD='some-password' FTP_PATH='/'
The FTP_PATH setting can be something else than the root of the FTP account. It can be any folder in the FTP space.
Update the software configuration :
command sed -i -e "s|[#]*\(.*BM_UPLOAD_FTP_USER=\).*$|\1\"${FTP_USER}\"|" \
-e "s|[#]*\(.*BM_UPLOAD_FTP_PASSWORD=\).*$|\1\"${FTP_PASSWORD}\"|" \
-e "s|[#]*\(.*BM_UPLOAD_FTP_HOSTS=\).*$|\1\"${FTP_HOST}\"|" \
-e "s|[#]*\(.*BM_UPLOAD_FTP_DESTINATION=\).*$|\1\"${FTP_PATH}\"|" \
'/etc/backup-manager.conf'
Enable the FTP export :
command sed -i -e "s/[#]*\(.*BM_UPLOAD_METHOD=\).*$/\1\"ftp\"/" \
'/etc/backup-manager.conf'
Copying a heavy folder to a remote FTP host
Using compression, even incremental, to backup more than a few Giga Bytes is unadapted. Backup Manager answer to this problem is to use "rsync" to backup heady data amounts. rsync is the best method to handle heavy data exchanges. Rsync use a SSH access to access a remote host.
If no SSH access is available, but a FTP account is provided (for example, a OVH dedicated server FTP backup space), rsync can not be used. The following method propose a way to solve this problem and allow to copy heavy amounts of data to a remote FTP account.
Create the folder for the specific scripts :
command mkdir --parent '/etc/backup-manager'
Download the scripts :
command wget 'https://raw.github.com/biapy/howto.biapy.com/master/backup-manager/ftp-copy.conf' \
--quiet --no-check-certificate --output-document='/etc/backup-manager/ftp-copy.conf'
command wget 'https://raw.github.com/biapy/howto.biapy.com/master/backup-manager/ftp-copy.sh' \
--quiet --no-check-certificate --output-document='/etc/backup-manager/ftp-copy.sh'
command chmod +x '/etc/backup-manager/ftp-copy.sh'
Install the needed tools :
command apt-get install yafc
Setup Backup Manager to run the ftp-copy script at the end of the backup process :
command sed -i -e "s|[#]*\(.*BM_POST_BACKUP_COMMAND=\).*$|\1\"/etc/backup-manager/ftp-copy.sh\"|" \
'/etc/backup-manager.conf'
The configuration of the script is simple. It use the Backup Manager FTP upload method parameters (see before).
Warning : Check that the Backup Manager configuration contain FTP access parameters.
Update "/etc/backup-manager/ftp-copy.conf" file, and add the paths to copy directly on the remote FTP in the FTP_COPY_FOLDERS setting, separated by spaces. For example :
export FTP_COPY_FOLDERS="/home/ftp /var/www/uploads /my/really/big/folder"
This script is a quick and dirty code. It can probably be upgraded.
Note : ftp-copy.sh use yafc FTP client and not ncftpput, because ncftpput has random connections errors when used with some FTP servers.
Thanks
- Thanks to Backup Manager developers.
- Thanks to GitHub for hosting the scripts used by this Backup Manager installation guide.
- Thanks to George Khaznadar for his work on backup-manager-tools and his patch for using SSH custom port with rsync..
- Thanks to prince_jammys on irc.freenode.net#bash for pointing me to How can I handle command-line arguments to my script easily? on Greg's Wiki.
- Thanks to Andy Skelton for Bash equivalent for PHP realpath().
