The Amazon EC2 toolkit works great, but it is a Java app which makes it an incredibly bloated way to gain access to a couple of command-line tools on a server. python-boto takes up all of about 1M, and its minimal Python dependencies were already installed on my Amazon instance. The necessary python script is very simple:
#!/usr/bin/env python thisVolume = 'vol-xxxxxxxx' waitSnapshot = 10 # wait increment (in seconds) while waiting for snapshot to complete print 'Logging into Amazon AWS....' from boto.ec2.connection import EC2Connection conn = EC2Connection(' ', ' ') print '' print 'Stopping MySQL....' import os os.system("/etc/init.d/mysql stop") print '' print 'Beginning backup of ' + thisVolume snapshot = conn.create_snapshot(thisVolume) newSnapshot = snapshot.id print 'Created new volume snapshot:', newSnapshot import time waitSnapshotTotal = waitSnapshot snapshot = conn.get_all_snapshots(str(newSnapshot)) print '' while snapshot.status != 'completed': print 'Snapshot status is ' + snapshot.status + ', ' \ 'wait ', waitSnapshotTotal, ' secs for the snapshot to complete before re-starting MySQL.' time.sleep(waitSnapshot) waitSnapshotTotal = waitSnapshotTotal + waitSnapshot snapshot = conn.get_all_snapshots(str(newSnapshot)) print snapshot print '' print 'Restarting MySQL....' os.system("/etc/init.d/mysql start")
The only gotcha was that the stale version of python-boto in Debian stable did not seem to have a "create_snapshot" function, so I just grabbed a newer version from Debian testing.
And of course, just call this script from cron daily to make backups automatic.
Amazon "Compute Cloud" services are setup such that storage for a running server is kept on an EBS ("Elastic Block Storage") volume mounted on the server. Long-term, offsite (geographic redundancy across multiple data centers) storage is kept in S3. There is a function to snapshot an EBS volume into S3.
List the volumes associated with your account:
This will also show what volumes are mounted on which servers. Stop the services running on the server that might be writing to the EBS (databases in particular, if you wish to have a usable backup):
/etc/init.d/apache2 stopCreate a snapshot:
ec2-create-snapshot vol-3159bd58Check status of snapshot:
SNAPSHOT snap-e309e38a vol-3159bd58 completed
To restore file(s) from one of the backup snapshots, simply create a new volume from the chosen snapshot. Attach it to instance of your running server. Mount the volume temporarily on the server and grab your files. Then unmount the volume, detach it, and delete it. A process that takes only a couple minutes, especially if you use the extremely convenient Firefox ElasticFox plugin.
Amazon's EC2 servers do not provide persistent storage. If the server is turned off or even perhaps halted, then all data since the server was initialized is lost. (I am pretty sure one can reboot an Amazon server without losing data, but have not tried....)
Therefore, at the least, all volatile data should be stored on an Amazon "Elastic Block Storage" (EBS) volume. As a further incentive, Amazon says that EBS is faster then the storage associated directly with an Amazon server, and that it is replicated across multiple servers in the data center. The downside, of course, is that you pay for EBS, both by gigabyte per month and by I/O per month.
Here is an excellent tutorial on how to use EBS with a MySQL database, and here is a nice general introduction.
I am going to move my Amazon Server's /var/www/ onto EBS. First use ec2-describe-instances to find out which Amazon zone the server is located in: us-east-1c. Now create a 5G EBS volume:
ec2-create-volume -z us-east-1c -s 5
From what I can see so far, changing the volume size probably involves snapshotting the volume, and then restoring the snapshot to a bigger volume. This small headache must be traded-off against the EBS cost of 10 cents per gigabyte per month.
ec2-describe-volumes will tell you when the new volume is available. Now attach the volume to the server instance at /dev/sdz:
ec2-attach-volume -d /dev/sdz -i i-c1a320a8 vol-3159bd58
Now login to the server and format the volume:
(Note that xfs has apparently been a popular format choice in the past because of its explicit freezability for snapshots, but I have seen in more then one place now that Amazon EC2 has problems with xfs.)
Setup /etc/fstab by adding the following entry:
/dev/sdz /vol ext3 noatime 0 0And mount the volume:
cd /Now move the contents of /var/www to the volume:
mv www /vol/
ln -s /vol/www/ www