EC2CloudBuild: Difference between revisions
Mparkachov (talk | contribs) m (Building OE images in EC2 cloud) |
(Undo revision 3267 by Yjytalamago (Talk)) |
(No difference)
| |
Latest revision as of 07:55, 24 November 2010
Prerequisites
In order to run instances (virtual machines) on an Amazon cloud infrastructure one need to get corresponding account and register on the AWS (Amazon Web Service). Getting Started Guide will help to get up and running, or basics could be checked on the Ubuntu clould guide.
The procedure uses Cloud Config utility from Ubuntu EC2 images to configure an instance and perform build. Here is an introduction of the utility with links to examples and documenation.
The build described here uses Ångström distribution and based on the setup-scripts described here.
Selecting AMI
The build was tested on 06 Septmeber 2010 with 32-bit Ubuntu 10.10 beta image, AMI ami-be6288d7. At the time of testing the 10.04 (LTS) image was not working due to the bug in shipped gzip. The status of the cloud config in the older versions of the ubuntu and debian is not clear. The list of the released images for ubuntu could be found here.
The procedure completely configures the image, so there is no need to use customized images or EBS-based images. The S3-based image provides more than enough "ephemerial" storage to perform the build.
Revisions
The test was performed for beagleboard machine with minimal-image. The build is based on 41569c5f4eb81baeb26f3a429602c8568d1c24fe commit, which corresponds to the testing_2010-08-23 branch.
Build procedure
- Sign in to the AWS Management Console
- Request either spot or on demand instance.
- Select AMI ami-be6288d7
- In the User data paste following cloud configuration data
#cloud-config
runcmd:
- dpkg-reconfigure -f noninteractive dash
- chown ubuntu /mnt
- [ su, -c, "( git clone git://gitorious.org/angstrom/angstrom-setup-scripts.git /mnt/oe )", ubuntu ]
- [ su, -c, "( cd /mnt/oe; sh oebb.sh config beagleboard )", ubuntu ]
- [ su, -c, "sed -i 's/^#PARALLEL/PARALLEL/g' /mnt/oe/build/conf/local.conf", ubuntu ]
- ln -s /mnt/oe /var/www/oe
- echo "<html><body><h2>Angstrom environment</h2><p>Browse building <a href='oe/'>directory</a></p><p>or watch <a href='oe/bitbake.log'>bitbake.log</a></p><br/><pre>---------------- local.conf ----------------" >/var/www/index.html
- cat /mnt/oe/build/conf/local.conf >>/var/www/index.html
- echo "</pre></body></html>" >>/var/www/index.html
- [ sh, -c, echo "========= Starting build =========" ]
- [ su, -c, "( cd /mnt/oe; sh oebb.sh update commit 41569c5f4eb81baeb26f3a429602c8568d1c24fe 2>&1 >/mnt/oe/update.log ; source ~/.oe/environment ; nohup bitbake minimal-image 2>&1 >/mnt/oe/bitbake.log & )", ubuntu ]
debconf_selections: |
debconf debconf/priority select critical
dash dash/sh boolean false
packages:
- sed
- wget
- cvs
- subversion
- git-core
- coreutils
- unzip
- texi2html
- texinfo
- libsdl1.2-dev
- docbook-utils
- gawk
- python-pysqlite2
- diffstat
- help2man
- make
- gcc
- build-essential
- g++
- desktop-file-utils
- chrpath
- libxml2-utils
- xmlto
- python-psyco
- apache2
- Submit request
After around 15 minutes required to initialize, start and install software one could point the browser to primitive page on the public DNS address of the instance to browse the build directory or get log of the bitbake.
The resulting machine deploy image could be downloaded directly from the browser, so no additional software is required.
Performance
The test was run on the c1.medium instance, which is correspondingly accessible and powerful enough. Spot instance was used to further reduce price of the build. The build was using parallel make with 2 threads and parallel bitbake with 2 threads as well.
The complete cycle from starting the instance to finished build took 2 hours 20 minutes.
The m1.small type of virtual machine proved to be underpowered (the build was cancelled after 5 hours).
The procedure retrieves quite a lot of data (sources) from the internet, but at the time of testing an incoming traffic was free and outgound traffic was negligibly small. After November 1, the inbound traffic will be priced as well, which will introduce additional costs. Caching of the downloaded files on an EBS volume could be considered in the future.
Changing targets
There are 3 important parameters to the build process, which could be changed.
- Machine architecture
Following line in the user data configures machine, which is in this case is beagleboard
sh oebb.sh config beagleboard
- State of OE repository
The commit could be specified to get repeatable tests. For the trunk build commit 41569c5f4eb81baeb26f3a429602c8568d1c24fe should be removed.
sh oebb.sh update commit 41569c5f4eb81baeb26f3a429602c8568d1c24fe
- Target image
The target image (in this case minimal-image) could be specified on the following line.
nohup bitbake minimal-image
Caution
Don't forget to terminate your EC2 instance after build, otherwise you'll be paying for running instance.