Z22SE.co.uk Backup Solution
OK, so I'm going to try and document exactly what is set up with regards to the backup solution we have in place.
vps.z22se.org.uk
The directory where the backup scripts are located
/root/scripts
We have two scripts that run the backups:
backup.sh (site files)
dbbackup.sh (databases)
backup.sh
#!/bin/bash
# Matt's script to copy site to Storage Server
# Location to store backups...
HOST="storage.z22se.org.uk"
PORT="8982"
USER="matt"
FORUM="/srv/samba/mattshare/Z22SE"
REM_NGINX="/srv/samba/mattshare/NGINX"
# Rsync options...
OPTS="-aruvP
--itemize-changes
--delete
--human-readable
--stats
--log-file=/root/scripts/backup.log"
LOCAL_FORUM="/home/nginx"
LOCAL_NGINX="/usr/local/nginx"
LOCAL_NGINX_BUP="/usr/local/nginxbackup"
/usr/bin/rsync -e "ssh -p ${PORT}" $OPTS $LOCAL_FORUM $USER@$HOST:$FORUM
/usr/bin/rsync -e "ssh -p ${PORT}" $OPTS $LOCAL_NGINX $LOCAL_NGINX_BUP $USER@$HOST:$REM_NGINX
In there, I'm using RSYNC to mirror the following directories across to the backup VPS
- /home/nginx
- /usr/local/nginx
- /usr/local/nginxbackup
These contain all the files needed to have the site back up and running, including the config files for nginx / php-fpm
They are being stored on the remote server here:
/srv/samba/mattshare
I have also set up logging of the RSYNC jobs, which outputs to:
/root/scripts/backup.log
For maintenance on the backup log, I've set this to be archived weekly using logrotate.
/etc/logrotate.d/backups
/root/scripts/backup.log {
weekly
rotate 10
compress
}
dbbackup.sh
#!/bin/bash
PASSWD="PASSWORD"
DIR="/root/scripts/databases"
SCRIPTS="/root/scripts"
USER="root"
TODAY=`date --date='today' +"%Y-%m-%d"`
# Dump the database(s)
/usr/bin/mysqldump --opt --single-transaction z22se_xenforo -u $USER -p$PASSWD > $DIR/z22seforum.$TODAY.sql
/usr/bin/mysqldump --opt z22se_newshop -u $USER -p$PASSWD > $DIR/z22seshop.$TODAY.sql
# Compress them with BZIP2
/usr/bin/bzip2 -f $DIR/*.$(/bin/date +%Y-%m-%d).sql
sleep 2
# Upload to backup server
/usr/bin/scp -P 8982 $DIR/*.$TODAY.sql.bz2 matt@storage.z22se.org.uk:/srv/samba/mattshare/Databases/
sleep 2
# Remove DB Files
rm $DIR/*.$TODAY.sql.bz2
This is pretty self explanatory. It's dumping both databases out to a .sql file. They are then compressed using bzip2. Once that is done, they are copied to the backup VPS using SCP. Finally, the dumped databases are removed from the server.
The scripts are ran from cron, which is configured in
/etc/crontab
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
HOME=/
# For details see man 4 crontabs
# Example of job definition:
# .---------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
# | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# | | | | |
# * * * * * user-name command to be executed
05 01 * * * root /root/scripts/backup.sh > /dev/null 2>&1
30 03 * * * root /root/scripts/dbbackup.sh > /dev/null 2>&1
backup.z22se.org.uk
This VPS has 500GB of disk space, so I've set it up to have 3 different stages of backup for the site.
The scripts used to move and store the archives are located here:
/home/matt/scripts
weekly.sh
monthly.sh
dbcleanup.sh
weekly.sh
#!/bin/bash
/usr/bin/rsync -aruvP --itemize-changes --delete --human-readable --stats --log-file=/home/matt/scripts/weekly.log /srv/samba/mattshare/Z22SE/ /srv/samba/mattshare/Z22SE_WEEKLY/
sshpass -p 'PASSWORD' /usr/bin/rsync -e "ssh -p 8982" -aruvP --itemize-changes --delete --human-readable --stats --log-file=/home/matt/scripts/weekly_nas.log /srv/samba/mattshare/Z22SE/ root@home.z22se.com:/mnt/soho_storage/samba/shares/Websites/Z22SE/
This mirrors the DAILY backup to a folder called WEEKLY. It also copies a version offsite to my NAS at home
monthly.sh
#!/bin/bash
/usr/bin/rsync -aruvP --itemize-changes --delete --human-readable --stats --log-file=/home/matt/scripts/monthly.log /srv/samba/mattshare/Z22SE/ /srv/samba/mattshare/Z22SE_MONTHLY/
Again, once a month, the DAILY folder is mirrored to a folder called MONTHLY, and the actions logged.
dbcleanup.sh
#!/bin/bash
OLD=`date --date='30 days ago' +"%Y-%m-%d"`
DIR="/srv/samba/mattshare/Databases"
/bin/rm -f $DIR/*.$OLD.sql.bz2
This is used to remove the old files which are more than 30 days old.
These are all executed as matt on the VPS via crontab
# Edit this file to introduce tasks to be run by cron.
#
# Each task to run has to be defined through a single line
# indicating with different fields when the task will be run
# and what command to run for the task
#
# To define the time you can provide concrete values for
# minute (m), hour (h), day of month (dom), month (mon),
# and day of week (dow) or use '*' in these fields (for 'any').#
# Notice that tasks will be started based on the cron's system
# daemon's notion of time and timezones.
#
# Output of the crontab jobs (including errors) is sent through
# email to the user the crontab file belongs to (unless redirected).
#
# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
#
# For more information see the manual pages of crontab(5) and cron(8)
#
# m h dom mon dow command
0 15 * * 0 /home/matt/scripts/weekly.sh
0 15 1 * * /home/matt/scripts/monthly.sh
0 04 * * * /home/matt/scripts/dbcleanup.sh