Howto CREATE BUNDLE UPLOAD and ACCESS custom Debian AMI using ubuntu

by

Introduction

This guide will provide all necessary steps on how to create, bundle, upload, run and connect Debian ETCH AMI on Amazon Elastic Compute Cloud (Amazon EC2). For this guide we have used a Ubuntu 9.04. However, any other Linux distribution can also be used as long as it contains java and ruby packages. For more information about Amazon EC2 read  here.

This page is not in any way an affiliate to Amazon Web Services. !

Prerequisites

  • Internet connection
  • registered user account for S3 and EC2 services with Amazon Web Services (AWS) 
  • Amazaon Access Key ID
  • Amazon Secret Access Key
  • Amazon Account Number
  • Amazon X.509 Certificate
  • at least 1GB free hard drive space
  • following packages need to be installed:
apt-get install ssh debootstrap ruby 
sun-java6-bin libopenssl-ruby curl

Before we start

 As you will see in the next sections of this guide many different files are required to successfully use Amazon’s EC2 Web Services. For the sake of simplicity, we will create a directory “aws” in ~/ and store all necessary files there for a quick access. There will be three exceptions:

  • AWS’s api and ami tools which we will install into /opt directory
  • chroot environment will be created in /chroot
  • Amazon’s account certificate and private key will be stored in ~/.ec2

Create Amazon Machine Image

Creating AMI in chroot environment

Create disk image with dd

To begin, we need to create a disk image of size appropriate for our installation. In this case we create a disk image around 750MB big. To do that we use dd command.

dd if=/dev/zero of=debian-ami count=750 bs=1M

Output of this command we create a file called debian-ami and it will be stored in our ~/aws directory.
Create disk image with dd

Make a filesystem on the disk image

Before we mount this image we need to create a file system. To do this job we can use mkfs.ext3 command as follows:

mkfs.ext3 -F debian-ami

You terminal output should be similar to one below:
Make a filesystem on the disk image

Mount newly created disk image

Now, we are almost ready to mount our new disk image. Before we do that, we need to decide where we would like to run chroot environment. In this guide we will use /chroot directory. Change to root ( super user ) and make directory with [[mkdir|mkdir]] command:

mkdir /chroot

Create chroot directory

to mount the disk image from our ~/aws directory we use following linux command:

mount -o loop /home/linuxconfig/aws/debian-ami /chroot

mount the disk image

Install debian into /chroot

To install Debian into /chroot we use debootstrap command which can be found on Debian as well as on Ubuntu. If you followed our prerequisites section the debootstrap command should be already available for you:

debootstrap --arch i386 etch /chroot/ http://ftp.debian.org

The output of this command will be quite long. The debootstrap will retrieve, validate, unpack and install all necessary packages. Install debian with debootstrap command

At the end you should get a messagesimilar to one shown on the next terminal screen shot:

successful chroot installation message

Configure chrooted Debian installation

Enter chrooted environment

Now that we have successfully installed minimal Debian system packages, we need to chroot into this installation and do some changes. Enter chroot environment with chroot command.

chroot /chroot

Enter chroot environment with chroot command

Create devices

mount /proc
 cd /dev
 MAKEDEV console
 MAKEDEV std

MAKEDEV create chrooted environment devices

Change root password

This will create new password for a super user account:
NOTE:We are still in the chroot environment !

passwd

create new password for a chroot super user account

Configure network interface

We need to edit network interfaces file to use DHCP on the boot. This command will do the trick:

echo -e 'auto lo\niface lo inet loopback\nauto eth0\niface eth0 inet dhcp' >> /etc/network/interfaces

configure network edit interfaces file

Amend /etc/fstab file

We also need to define some mount points:

echo -e '/dev/sda1 / ext3 defaults 0 1\n/dev/sda2 swap swap defaults 0 0' > /etc/fstab

edit fstab file

Install sshd

Once we would have our new AMI ready, uploaded and started we would connect to it via ssh. Therefore, we need to install ssh daemon. Use apt-get to install ssh package:
NOTE: We are still in chrooted environment

apt-get install ssh

Your chroot environment is sharing the same Internet connection with your host so everything should go smoothly. Do not worry about “Setting locale failed.” warring messages.

Exit / umount chroot environment

All should be ready, so we can exit chroot environment:

exit

and use unmount to unmount file image:

umount -l /chroot

Upload New Amazon Machine Image

Setup Amazon environment variables, keys and cert’s

Now it is time that we extract our account details and certificates from the amazon web site. Create ~/.ec2 directory and save there your certificates. The steps involved are described [http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=84 here]. Navigate to GET STARTED -> Setting up an Account. If you have private key and certificate saved, we can set environmental variable, so we do not have to refer to them with a full path when using ami and api tools:

mkdir ~/.ec2
export EC2_PRIVATE_KEY=~/.ec2/pk-K5AHLDNT3ZI28UIE6Q7CC3YZ4LIZ54K7.pem
export EC2_CERT=~/.ec2/cert-K5AHLDNYYZI2FUIE6R7CC3YJ4LIZ54K7.pem

EC2 AMI Tools and EC2 API Tools are based on java. Set environment variable for java and confirm that java is installed:

export JAVA_HOME=/usr/
$JAVA_HOME/bin/java -version

Setup java environmental variable As a last thing we can also setup account number variable, access key and secret key:
NOTE: Access key, secret key and account number are randomly created for this guide to fit a real format. They are not valid! However, if you have plenty time you may try !

export EC2_ACCNO=155678941235
export ACCESS_KEY=1WQ6FJKYHJMPTJ3QR6G2
export SECRET_KEY=VDYxRzosnDWvxrJ97QntVpsSUBAavGHE1QJELEyY

Setup EC2 AMI Tools from Amazon S3

This part of this tutorial will explain how to setup and use EC2 AMI Tools in order to bundle and upload new AMI.

Download EC2 AMI Tools

Download ami tools :

cd ~/aws
 wget http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.zip

Download amazon ami tools zip file

Install EC2 AMI Tools

unzip ec2-ami-tools.zip to /opt:
NOTE: Use sudo or switch to root !

unzip -d /opt/ ec2-ami-tools.zip

Include ami tools to the PATH variable and EC2_HOME:

export PATH=$PATH:/opt/ec2-ami-tools-1.3-21885/bin
export EC2_HOME=/opt/ec2-ami-tools-1.3-21885

Export ami tools variables

Bundle new AMI

All is set up and we are ready to bundle our new Debian AMI. You will be asked “Please specify a value for arch [i386]”, if left blank default is 10MB:

ec2-bundle-image -i debian-ami --cert $EC2_CERT --privatekey $EC2_PRIVATE_KEY -u $EC2_ACCNO

Bundle AMI image

Upload AMI files

Previously, bundle image ami tool will create files in /tmp directory by default . This is also the place where your XML manifest for your new AMI is located. Now upload AMI:
NOTE:If bucket does not exist it will be created! Moreover, you MUST choose your own name for the bucket.

ec2-upload-bundle -b linux-debian-etch -m /tmp/debian-ami.manifest.xml -a $ACCESS_KEY -s $SECRET_KEY

Upload AMI with ec2-upload-bundle tool

Setup EC2 API Tools from Amazon S3

This part of this tutorial will explain how to setup and use EC2 API Tools in order to register and use new AMI.

Download EC2 API Tools

Download api tools :

cd ~/aws
wget http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip

Download API tools

Install EC2 API Tools

unzip ec2-api-tools.zip to /opt:
NOTE: Use sudo or switch to root !

unzip -d /opt/ ec2-api-tools.zip

Include api tools to the PATH variable and EC2_HOME:

export PATH=$PATH:/opt/ec2-api-tools-1.3-24159/bin/
export EC2_HOME=/opt/ec2-api-tools-1.3-24159/

install amazon api tools

Register AMI with API tools

At this stage we are ready to register our new AMI. After registering, we will get AMI’s id number.
NOTE:For an Amazon API tools, the path to your amazon EC2 certificate and private key are automatically extracted from the environment variables defined earlier.

ec2-register linux-debian-etch/debian-ami.manifest.xml

Register AMI with ec2-register

Run AMI instance

Now that we have got a AMI’s registered number, we can start it:

ec2-run-instances ami-b9f115d0

Start amazon AMI instance

Describe AMI instance

Well, AMi is running and we need to know some more information about it such as IP address or full domain name, use the instance number generated when starting AMI. ( see previous step !):

ec2-describe-instances i-c369ccaa

Describe amazon machine instance

Connecting to AMI with ssh

If this is not your first AMI you probably already have your port 22 enabled. If not, run this command first before you attempt to connect to it:

ec2-authorize default -p 22

Once enabled, use shh command to connect to your new Debian ETCH AMI:

ssh root@IP-address or full-domain-name

NOTE:We retrieved the full domain name previously with ec2-describe-instances command.
Connect to AMI via ssh and enable port 22

Appendix

Other useful EC2 commands

Console output

To see what is happening with our instance, we can use ec2-get-console-output with combination of our instance ID:

ec2-get-console-output

Shut down the Amazon EC2 instance

To shut down the Amazon EC2 instance use:

ec2-terminate-instances

View the list of registered AMI’s

To view the list of your registered Amazon Machine Images:

ec2-describe-images

Related Linux Tutorials: