Using Resin with Amazon EC2
Last week I was been looking at ways to take advantage of Resin 4.0’s new cloud features within Amazon EC2. I’m in the process of developing some tools to make this simpler, but in the meantime, it’s already possible to get up and running with Resin 3.1 on Amazon EC2. In this post, I’ll show you how to create an Amazon Machine Image (AMI) with Resin and install WordPress on Quercus with MySQL so that your data persists on the Elastic Block Store.
Prerequisites
You’ll need to have a basic understanding of Amazon EC2. Specifically, you’ll need to know how to launch an instance, create and install a key pair on a running instance, and ssh into instance. You’ll also need to have installed the EC2 AMI and API tools. Many of the steps below can also be performed from the command line or the online AWS Management Console. This post will focus on installing on an Ubuntu image, but other Linux/Unix distributions should be similar. A lot of the steps that I give below are synthesized from several sources, so if you need more depth, check them out:
- AWS Developers Guide: Creating an AMI
- Alestic Debian/Ubuntu AMI build script
- Running MySQL on Amazon EC2 with Elastic Block Store
As a final note before we begin, remember that starting new instances and using storage like EBS or S3 will incur charges on your AWS account.
Launching a bootstrap instance
The first four of these steps can be done from the online AWS Management Console instead of the command line.
- On your local machine, create a new key pair for your AMI:
local> ec2-create-keypair resin-keypair
Save the generated key (the part of the output starting with the “—–BEGIN RSA PRIVATE KEY—–” and ending with “—–END RSA PRIVATE KEY—–“) to a file named id_rsa-resin-keypair.
- Start a new instance using this key pair. I used a public 32-bit Ubuntu 8.04 AMI with the following id:
local> ec2-run-instances ami-71fd1a18 -k resin-keypair
- Wait a few moments for the machine to start up. If you noted the instance id output by the last command, you can monitor the progress with this command:
local> ec2-describe-instances <instance-id>
Once it says “running”, you’re ready for the next step. Note the randomly assigned domain name of the instance. It should look something like “ec2-123-45-67-89.compute-1.amazonaws.com”.
- Make sure that you can access the SSH port by authorizing access to it:
local> ec2-authorize default -p 22
- Copy the following files to your running instance’s /tmp directory using scp:
- Your private key and certificate
- Your Resin Professional license, if you have one
- These sample scripts
- The Alestic Ubuntu AMI build script
- Login to your running instance using the key pair you generated earlier:
local> ssh -i id_rsa-resin-keypair root@<instance IP>
Building the AMI
- On the running instance, change to the /tmp directory and untar the scripts mentioned above:
ec2-instance /tmp> tar zxvf ec2resin1.tar.gz
ec2resin-wordpress-script
init.resin
wp-config.php - If you have a Resin Professional license, edit ec2resin-wordpress-script and enter the filename of the license as the value of the LICENSE variable, e.g.:
LICENSE=/tmp/100XXXX.license
- Edit the Alestic script to remove the –generate-fstab argument from the ec2-bundle-vol invocation near the
bottom of the ec2ubuntu-build-ami file so that the end looks like this:chroot $imagedir ec2-bundle-vol \ -r $bundlearch \ -d /tmp \ -p $prefix \ -u $AWS_USER_ID \ -k /tmp/pk.pem \ -c /tmp/cert.pem \ -s $size \ -e /tmp \ $bundle_vol_options ec2-upload-bundle \ --retry \ -b $bucket \ -m $imagedir/tmp/$prefix.manifest.xml \ -a $AWS_ACCESS_KEY_ID \ -s $AWS_SECRET_ACCESS_KEY \ --location $location umount $imagedir/dev/pts umount $imagedir/proc || true set +x cat <<EOF Now you might want to run this command: ec2-register $bucket/$prefix.manifest.xml EOF
- Run the ec2ubuntu-build-ami script, filling in the values shown below:
ec2-instance /tmp> ./ec2ubuntu-build-ami --codename hardy \ --bucket <your-s3-bucket> \ --prefix index \ --user <your-user-id> \ --access-key <your-access-key> \ --secret-key <your-secret-key> \ --private-key /tmp/<your-private-key>.pem \ --cert /tmp/<your-certificate>.pem \ --package ubuntu-minimal \ --package ubuntu-standard \ --package build-essential \ --package libssl-dev \ --package mysql-server \ --package postfix \ --script /tmp/ec2resin-wordpress-scriptMake sure that you list the full path of your private key, certificate, and the ec2resin-wordpress-script. They should all be in /tmp. The S3 bucket will be created for you if it doesn’t already exist. Make sure that the S3 bucket name isn’t already taken. This script takes a long time and you wouldn’t want to get to the end, only to discover that it has failed because the bucket was taken. The reason that it takes so long is that it’s building a complete Ubuntu image, pretty much from scratch. You’ll be asked to agree to the Sun Java license along the way, so you’ll need to agree to that to proceed.
- Once the AMI is built and uploaded by the script, you’ll need to register it to be able to use it. Use either the ec2-register command shown by the script or the AWS Management Console. Don’t start your new AMI yet.
Preparing the database
- From your local machine or the AWS Management Console, create a new EBS volume. The following command creates a volume with 10GB, but you can adjust that to your needs:
local> ec2-create-volume -z us-east-1a -s 10
- Attach the volume to your running bootstrap instance. You can do this either with the AWS management console or the ec2-attach-volume command. Attach the volume at /dev/sdh, e.g.:
local> ec2-attach-volume -d /dev/sdh -i i-XXXXXXXX vol-XXXXXXXX
- From the bootstrap instance that you’ve started, install the XFS tools and MySQL server:
ec2-instance> apt-get update
ec2-instance> apt-get install xfsprogs mysql-serverSet the root password to empty for now.
- Create the filesystem on the EBS volume:
ec2-instance> mkfs.xfs /dev/sdh
ec2-instance> echo “/dev/sdh /vol xfs noatime 0 0″ >> /etc/fstab
ec2-instance> mount /vol
ec2-instance> mkdir -p /vol/lib /vol/log - Stop the MySQL server:
ec2-instance> /etc/init.d/mysql stop
ec2-instance> killall mysqld_safe - Setup MySQL to read the database from the EBS volume:
ec2-instance> mv /var/lib/mysql /vol/lib/
ec2-instance> mv /var/log/mysql /vol/log/
ec2-instance> cat > /etc/mysql/conf.d/mysql-ec2.cnf <<EOM
[mysqld]
innodb_file_per_table
datadir = /vol/lib/mysql
log_bin = /vol/log/mysql/mysql-bin.log
max_binlog_size = 1000M
#log_slow_queries = /vol/log/mysql/mysql-slow.log
#long_query_time = 10
EOM
ec2-instance> rsync -aR /etc/mysql /vol/ - Start the MySQL server again and create a database for WordPress:
ec2-instance> /etc/init.d/mysql start
ec2-instance> mysql -u root
mysql> CREATE DATABASE wordpress; - Stop the MySQL server and unmount the EBS volume:
ec2-instance> /etc/init.d/mysql stop
ec2-instance> umount /vol - From your local machine or the AWS Management Console, detach the EBS volume:
ec2-detach-volume -i i-XXXXXXXX vol-XXXXXXXX
Start up a new instance
- Using the AMI ID that you registered earlier, start a new instance:
local> ec2-run-instances ami-XXXXXXXX -k resin-keypair
- Once the instance has started, attach the EBS volume:
local> ec2-attach-volume -d /dev/sdh -i i-XXXXXXXX vol-XXXXXXXX
- Log into the instance and mount the EBS volume:
ec2-instance2> mount /vol
- Start MySQL on the new instance:
ec2-instance2> /etc/init.d/mysql start
- Finally… Browse to the instance and walk through WordPress’ easy install.
Conclusion
The above instructions are just an example of how to create a clean Ubuntu AMI for WordPress, but you can extend this approach to any Java or PHP application. Just edit the ec2resin-wordpress-script and put in your custom installation instructions. At the moment, EC2 doesn’t allow you to attach EBS volumes at instance startup, but the AWS developers are aware of the problem and will probably have it available soon. If that becomes possible, it will make starting up an instance easier. The hard work will only be in the AMI build.
Please let me know if you give this a try. I’d love to see what you come up with. There are already a few users installing Resin on EC2 instances, but of course we’d like to see more. 