I’m hosting all my stuff on a single EC2 instance (a Micro, to be precise) and so far it’s been working out great. People are complaining about EC2 performance, particularly when combined with EBS volumes, but I have to say that even this ridiculously small server beats every single shared hosting provider I had before. NginX + PHP + APC + Memcached = awesome!
However, I did manage to cause a significant outage for a few yours yesterday. See, my backup strategy is somewhat cowboy-style. While there are offsite backups, those are few and far between. So if anything goes wrong, there is a huge incentive to not restore from backup if at all possible. And yesterday, something finally did go wrong. After a routine upgrade of the Linux distro powering my t1.micro instance, the machine simply refused to come back up after reboot.
There is little one can do in this case, when an instance simply refuses to boot into an SSH-able state. In the end, I rebuilt the server from a blank new t1.micro instance and copied over just the data from the old system volume. This is where EC2 is amazing: it allows you to attach any EBS volume to any instance, even while it’s running. Thus, the need to restore from a well-aged backup was once again averted. But the length of this outage was completely avoidable. Things to consider when you have a setup similar to mine:
- Keep a spare bootable instance ready, in case your primary fails. EC2 instances only cost money when they run, so you should be fine keeping a deactivated spare around.
- Always resync your backup before upgrading your distro. (duh!)
- You can always attach your IP addresses to a spare instance in seconds.
- A two-server setup with failover is probably a good idea.