Author: Lubos Rendek

1. Introduction

This guide will provide all necessary steps on how to create, bundle, upload, run and connect Debian ETCH AMI on Amazon Elastic Compute Cloud (Amazon EC2). For this guide we have used a Ubuntu 9.04. However, any other Linux distribution can also be used as long as it contains java and ruby packages. For more information about Amazon EC2 read  here.

This page is not in any way an affiliate to Amazon Web Services. !

2. Prerequisites

• Internet connection
• registered user account for S3 and EC2 services with Amazon Web Services (AWS) 
• Amazaon Access Key ID
• Amazon Secret Access Key
• Amazon Account Number
• Amazon X.509 Certificate
• at least 1GB free hard drive space
• following packages need to be installed:

apt-get install ssh debootstrap ruby 
sun-java6-bin libopenssl-ruby curl 

3. Before we start

 As you will see in the next sections of this guide many different files are required to successfully use Amazon's EC2 Web Services. For the sake of simplicity, we will create a directory "aws" in ~/ and store all necessary files there for a quick access. There will be three exceptions:

• AWS's api and ami tools which we will install into /opt directory
• chroot environment will be created in /chroot
• Amazon's account certificate and private key will be stored in ~/.ec2

4. Create Amazon Machine Image

4.1. Creating AMI in chroot environment

4.1.1. Create disk image with dd

To begin, we need to create a disk image of size appropriate for our installation. In this case we create a disk image around 750MB big. To do that we use dd command.

dd if=/dev/zero of=debian-ami count=750 bs=1M 

Output of this command we create a file called debian-ami and it will be stored in our ~/aws directory.

4.1.2. Make a filesystem on the disk image

Before we mount this image we need to create a file system. To do this job we can use mkfs.ext3 command as follows:

mkfs.ext3 -F debian-ami 

You terminal output should be similar to one below:

4.1.3. Mount newly created disk image

Now, we are almost ready to mount our new disk image. Before we do that, we need to decide where we would like to run chroot environment. In this guide we will use /chroot directory. Change to root ( super user ) and make directory with [[mkdir|mkdir]] command:

mkdir /chroot 

to mount the disk image from our ~/aws directory we use following command:

mount -o loop /home/linuxconfig/aws/debian-ami /chroot 

4.1.4. Install debian into /chroot

To install Debian into /chroot we use debootstrap command which can be found on Debian as well as on Ubuntu. If you followed our prerequisites section the debootstrap command should be already available for you:

debootstrap --arch i386 etch /chroot/ http://ftp.debian.org 

The output of this command will be quite long. The debootstrap will retrieve, validate, unpack and install all necessary packages.

At the end you should get a messagesimilar to one shown on the next terminal screen shot:

4.2. Configure chrooted Debian installation

4.2.1. Enter chrooted environment

Now that we have successfully installed minimal Debian system packages, we need to chroot into this installation and do some changes. Enter chroot environment with chroot command.

chroot /chroot 

4.2.2. Create devices

mount /proc
 cd /dev
 MAKEDEV console
 MAKEDEV std 

4.2.3. Change root password

This will create new password for a super user account:
NOTE:We are still in the chroot environment !

passwd 

4.2.4. Configure network interface

We need to edit network interfaces file to use DHCP on the boot. This command will do the trick:

echo -e 'auto lo\niface lo inet loopback\nauto eth0\niface eth0 inet dhcp' >> /etc/network/interfaces 

4.2.5. Amend /etc/fstab file

We also need to define some mount points:

echo -e '/dev/sda1 / ext3 defaults 0 1\n/dev/sda2 swap swap defaults 0 0' > /etc/fstab 

4.2.6. Install sshd

Once we would have our new AMI ready, uploaded and started we would connect to it via ssh. Therefore, we need to install ssh daemon. Use apt-get to install ssh package:
NOTE: We are still in chrooted environment

apt-get install ssh 

Your chroot environment is sharing the same Internet connection with your host so everything should go smoothly. Do not worry about "Setting locale failed." warring messages.

4.2.7. Exit / umount chroot environment

All should be ready, so we can exit chroot environment:

exit 

and use unmount to unmount file image:

umount -l /chroot 

5. Upload New Amazon Machine Image

5.1. Setup Amazon environment variables, keys and cert's

Now it is time that we extract our account details and certificates from the amazon web site. Create ~/.ec2 directory and save there your certificates. The steps involved are described [http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=84 here]. Navigate to GET STARTED -> Setting up an Account. If you have private key and certificate saved, we can set environmental variable, so we do not have to refer to them with a full path when using ami and api tools:

mkdir ~/.ec2
export EC2_PRIVATE_KEY=~/.ec2/pk-K5AHLDNT3ZI28UIE6Q7CC3YZ4LIZ54K7.pem
export EC2_CERT=~/.ec2/cert-K5AHLDNYYZI2FUIE6R7CC3YJ4LIZ54K7.pem 

EC2 AMI Tools and EC2 API Tools are based on java. Set environment variable for java and confirm that java is installed:

export JAVA_HOME=/usr/
$JAVA_HOME/bin/java -version 

As a last thing we can also setup account number variable, access key and secret key:
NOTE: Access key, secret key and account number are randomly created for this guide to fit a real format. They are not valid! However, if you have plenty time you may try !

export EC2_ACCNO=155678941235
export ACCESS_KEY=1WQ6FJKYHJMPTJ3QR6G2
export SECRET_KEY=VDYxRzosnDWvxrJ97QntVpsSUBAavGHE1QJELEyY 

5.2. Setup EC2 AMI Tools from Amazon S3

This part of this tutorial will explain how to setup and use EC2 AMI Tools in order to bundle and upload new AMI.

5.2.1. Download EC2 AMI Tools

Download ami tools :

cd ~/aws
 wget http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.zip 

5.2.2. Install EC2 AMI Tools

unzip ec2-ami-tools.zip to /opt:
NOTE: Use sudo or switch to root !

unzip -d /opt/ ec2-ami-tools.zip 

Include ami tools to the PATH variable and EC2_HOME:

export PATH=$PATH:/opt/ec2-ami-tools-1.3-21885/bin
export EC2_HOME=/opt/ec2-ami-tools-1.3-21885 

5.2.3. Bundle new AMI

All is set up and we are ready to bundle our new Debian AMI. You will be asked "Please specify a value for arch [i386]", if left blank default is 10MB:

ec2-bundle-image -i debian-ami --cert $EC2_CERT --privatekey $EC2_PRIVATE_KEY -u $EC2_ACCNO 

5.2.4. Upload AMI files

Previously, bundle image ami tool will create files in /tmp directory by default . This is also the place where your XML manifest for your new AMI is located. Now upload AMI:
NOTE:If bucket does not exist it will be created! Moreover, you MUST choose your own name for the bucket.

ec2-upload-bundle -b linux-debian-etch -m /tmp/debian-ami.manifest.xml -a $ACCESS_KEY -s $SECRET_KEY 

5.3. Setup EC2 API Tools from Amazon S3

This part of this tutorial will explain how to setup and use EC2 API Tools in order to register and use new AMI.

5.3.1. Download EC2 API Tools

Download api tools :

cd ~/aws
wget http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip 

5.3.2. Install EC2 API Tools

unzip ec2-api-tools.zip to /opt:
NOTE: Use sudo or switch to root !

unzip -d /opt/ ec2-api-tools.zip 

Include api tools to the PATH variable and EC2_HOME:

export PATH=$PATH:/opt/ec2-api-tools-1.3-24159/bin/
export EC2_HOME=/opt/ec2-api-tools-1.3-24159/ 

5.3.3. Register AMI with API tools

At this stage we are ready to register our new AMI. After registering, we will get AMI's id number.
NOTE:For an Amazon API tools, the path to your amazon EC2 certificate and private key are automatically extracted from the environment variables defined earlier.

ec2-register linux-debian-etch/debian-ami.manifest.xml 

5.4. Run AMI instance

Now that we have got a AMI's registered number, we can start it:

ec2-run-instances ami-b9f115d0 

5.5. Describe AMI instance

Well, AMi is running and we need to know some more information about it such as IP address or full domain name, use the instance number generated when starting AMI. ( see previous step !):

ec2-describe-instances i-c369ccaa 

5.6. Connecting to AMI with ssh

If this is not your fist AMI you probably already have your port 22 enabled. If not, run this command first before you attempt to connect to it:

ec2-authorize default -p 22 

Once enabled, use shh command to connect to your new Debian ETCH AMI:

ssh root@IP-address or full-domain-name 

NOTE:We retrieved the full domain name previously with ec2-describe-instances command.

6. Appendix

6.1. Other useful EC2 commands

6.1.1. Console output

To see what is happening with our instance, we can use ec2-get-console-output with combination of our instance ID:

ec2-get-console-output  

6.1.2. Shut down the Amazon EC2 instance

To shut down the Amazon EC2 instance use:

ec2-terminate-instances  

6.1.3. View the list of registered AMI's

To view the list of your registered Amazon Machine Images:

ec2-describe-images