“A pint of sweat, saves a gallon of blood.” — George S. Patton

You know you should do it, but you still don’t. It’s eating at you really. Always in the back of your mind, that nagging thought — what would happen if I lost all my data?

So here you go. My recipe for taking nightly snapshots of mission critical data and backing up to Amazon S3. Just replace the relevant directories, MySQL password, and S3 bucket name with your own. You’ll also need to setup your AWS SSH keys before s3cmd will work.

Note: this script requires the program s3cmd which can be installed on debian-based systems (like Ubuntu) with apt-get -y install s3cmd

GIT_DIR="/srv/git"
HUDSON_DIR="/srv/hudson"
REDMINE_DIR="/mnt/redmine"
BACKUP_DIR="/root/backups"


DATE_TAG=$(date +%m_%d_%Y)

backup key directories

tar pczvf $BACKUP_DIR/git.$DATE_TAG.tgz $GIT_DIR
tar pczvf $BACKUP_DIR/hudson.$DATE_TAG.tgz $HUDSON_DIR
tar pczvf $BACKUP_DIR/redmine.$DATE_TAG.tgz $REDMINE_DIR

#

backup databases

mysqldump --password=your_pw_here --all-databases | gzip -c > $BACKUP_DIR/mysql
.$DATE_TAG.gz

for f in $BACKUP_DIR/$DATE_TAG
do
    /usr/bin/s3cmd -c /root/.s3cfg put $f s3://your_bucket_here.com/$(basename $f) >> 
/root/backup.log 2>&1
done

Save that script to a file named backups and drop it in /etc/cron.daily. Make sure it’s root-owned (chown root.root backups) and has the proper permissions (chmod 755 backups).

Keep in mind this will accumulate files in the backups directory in its current state, so you might want to add a line to remove the backups after storing to s3. Personally I like to delete them manually after ensuring the backup was performed correctly.

I hope this saves somebody’s ass someday — it’s already saved mine.

Tags: amazon, aws, backups, blueprints, s3, sysadmin

This entry was posted on Monday, February 16th, 2009 at 8:58 pm and is filed under general. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.