Buildbot and EC2

Project Name

Buildbot and EC2

Project Description

The BuildBot is software for automatically building and testing other software. EC2 is Amazon's "Elastic Compute Cloud", a service that lets you run Virtual Machines on Amazon's computing grid using a web API. It would be awesome if BuildBot had an EC2 module, and could spawn Virtual Machines on-demand to perform tasks. Conveniently, BuildBot is written in Python, and there's a Python library for interfacing with EC2.

References: Amazon EC2 Basics For Python Programmers

Project Leader(s)

Adam Delyea

Project Contributor(s)

Dominic Baranski

Project Details

Release 0.1

   Buildbot and EC2/0.1

Release 0.2

   Buildbot and EC2/0.2

Release 0.3

   Buildbot and EC2/0.3

Release 0.4

   Buildbot and EC2/0.4

Release 0.5

   Buildbot and EC2/0.5

Tasks

Task	Details	Priority	Contributors	Status	Target	Completed	Link(s)
Downloadable Try Builds	Allow users to download their completed builds	Medium	Adam Delyea	In Progress	03/02/08		Mozilla@Seneca Cluster Administration
Try Server Apache Authentication	Authenticate users using the Apache config	Medium	Adam Delyea	In Progress	03/02/08		Mozilla@Seneca Cluster Administration
Seneca EC2 Account	Set up an account for Seneca on Amazon's EC2 Service	Medium	Adam Delyea	In Progress	03/02/08		Mozilla@Seneca Cluster Administration
Try Server @ Seneca	Port Try Server from external server to internal servers	High	Adam Delyea David Humphrey	In Progress - Dave is in the process of setting up access to the Mozilla@Seneca cluster	01/02/08		Mozilla@Seneca Cluster Administration
Try Server	Start a Try Server on a Linux machine.	High	Adam Delyea	Done	14/01/08	14/01/08	Mozilla Try Server
EC2 on Try Server	Make the Try Server use EC2 to spawn build slaves.	High	Adam Delyea	Done	14/01/08	14/01/08
Define new build step	Add a new build step to the build master config file. The build step will be in an external file and imported into the build master config file.	High	Adam Delyea	Done			Writing New Build Steps
Connect To EC2 using Python	Utilize the EC2 python library to connect to an account on the EC2 network. Once connected, display some information about the users account to prove it's actually connecting.	High	Adam Delyea	Done			Boto Amazon Web Services Library in Python EC2 hearts Python
Start new instances	Allow Buildbot to spawn an instance on demand. This could possibly be a function called right before the build slave list is populated, and would pass back an array of build slaves.	High	Adam Delyea	Done
Add new instances to build slave list	Allow newly created EC2 instances to be added to the build slave list. After looking more closely at how the buildslave is created, this goal doesn't make any sense, since this is not really how the buildslave list works.	Cut	Adam Delyea	Cut
Start a new build on an EC2 instance	Allow Buildbot to successfully start a build using a build slave on an EC2 instance	High	Adam Delyea	Done
Load a script on startup in Linux	Make it possible to have a script load when Linux starts	High	Adam Delyea	Done
Testing	People needed to download the revisions and test them to make sure it works in different environments. Note, this will require you to have your own EC2 account.	Expired	NEEDED! *** Dominic Baranski	Expired
Different Builds	Need different projects to try and build, to make sure it works to build projects other than Mozilla	Expired	NEEDED! *** Dominic Baranski	Expired
Build Mozilla using an EC2 Instance	Allow Buildbot to successfully build mozilla using a build slave on an EC2 instance	High	Adam Delyea	Done
Complete Documentation		Medium	Adam Delyea	In Progress
Generic EC2 instance for Buildbot Slaves	Making a generic EC2 Instance for Buildbot	Medium	Adam Delyea	Done
Startup Scripts for making your own EC2 instances		Medium	Adam Delyea	Done

Contributions

 
*** Would you like to contribute? Well here's an excellent opportunity for you to do just that! Feel free to e-mail me at <awdelyea@gmail.com>
 

Project Resources

• Mozilla Buildbot links: docs and code
• EC2 Firefox Extension
• Amazon-ec2-for-people-who-prefer-debian-and-python-over-fedora-and-java

Project News

For a less formal, and more up-to-date view on whats happening, see the BLOG.

2007-09-17: Project claimed!

2007-10-11: Found a python EC2 library that has all of the functions I will need for interacting with EC2. Just need to write a custom build step to run these library functions.

2007-10-12: Release v0.1 is complete and available for download.

2007-10-12: Release v0.2 loosely defined. More detail to follow.

2007-11-17: Release v0.2 is complete and available for download: can start an instance and start an EC2BuildSlave

2007-11-21: Release v0.2.1 is complete and available for download: now to include variable EC2 users, and instances, and DNS'

2007-12-13: Release v0.3 is complete and available for download: easier to install and setup, can use to build projects now

2008-12-14: Release v0.4 Defined. In the process of getting accounts and machines for Try Server Farm.