Press enter to see results or esc to cancel.

Backing up to Amazon S3

Even though you’re backing up your files on additional hard drive, optical media or something else that you store in a drawer of your desk you’re not quite safe yet. It’s essential to do backups to remote location once in a while. At a price of $0.095 per GB per month Amazon’s S3 storage seems quite an interesting option. To make things easier there is a command-line tool called ‘aws‘ which enables you to use Amazon’s Web Services simply by issuing a few commands.

To backup my development server with Git repositories  and stuff like that I’ve written a simple Bash script:

#!/bin/bash

EC2_ACCESS_KEY=YOUR_ACCESS_KEY
EC2_SECRET_KEY=YOUR_SECRET_ACCESS_KEY
BUCKET_NAME="NAME_OF_THE_BUCKET"
MYSQL_PASSWORD="MYSQL_ROOT_PASSWORD"
NOW=$(date +"%Y-%m-%d_%H-%M")
TMPDIR="/var/backups/$NOW/" #temporary directory for storing archives

add_md5_hash() {
  if [ -z "$1" ]
  then
    echo "You must pass filename."
    return -1
  fi
  echo "Calculating hash..."
  md5sum $1 >> "$1.md5"
  echo "Uploading hash..."
  s3put "$BUCKET_NAME/$1.md5" "$1.md5"
}

backup_dir() {
  if [ -z "$1" ]
  then
    echo "You must pass directory name."
    return -1
  fi
  DIR="$1"
  FILENAME="$NOW${1//\//_}.tar.gz"

  echo "Backing up $1"
  mkdir -p "$TMPDIR"
  cd "$TMPDIR"
  echo "Creating temporary archive $FILENAME"
  tar -zcf "$FILENAME" "$1"
  echo "Uploading to AWS ( $BUCKET_NAME )"
  s3put "$BUCKET_NAME/$FILENAME" "$FILENAME"
  add_md5_hash $FILENAME
}

backup_mysql_database() {
  mkdir -p "$TMPDIR"
  cd "$TMPDIR"
  FILENAME="${NOW}_db_$1.sql"
  echo "Creating temporary file $FILENAME"
  mysqldump -u root -p$MYSQL_PASSWORD --database $1 >> $FILENAME
  echo "Uploading to AWS ( $BUCKET_NAME )"
  s3put "$BUCKET_NAME/$FILENAME" "$FILENAME"
  add_md5_hash $FILENAME
}

backup_mysql_server() {
  mkdir -p "$TMPDIR"
  cd "$TMPDIR"
  FILENAME="${NOW}_localhost.sql"
  echo "Creating temporary file $FILENAME"
  mysqldump -u root -p$MYSQL_PASSWORD --all-databases >> $FILENAME
  echo "Uploading to AWS ( $BUCKET_NAME )"
  s3put "$BUCKET_NAME/$FILENAME" "$FILENAME"
  add_md5_hash $FILENAME
}

#Examples:
backup_dir "/PATH/TO/THE/DIRECTORY/YOU/WANT/TO/BACKUP"
backup_mysql_server #if you want to backup all databases
backup_mysql_database "DB_NAME" #if you want to backup single database

For this script to work you need to install aws. Before using this script you also have to adjust settings at the top and put appropriate backup commands at the end of file, for example:

backup_dir "/PATH/TO/THE/DIRECTORY/YOU/WANT/TO/BACKUP"

to backup a directory. Or

backup_mysql_server

to backup mysql server at localhost. Or

backup_mysql_database "DB_NAME"

to backup a single database.

Make sure that only appropriate users get read access to this file because of delicate information it contains. After initial configuration you can configure Cron to run this script at regular intervals.

I hope you find this post helpful. Now get back to coding.

Enhanced by Zemanta