Changes

Jump to: navigation, search

Buildbot and EC2

471 bytes removed, 11:23, 26 March 2008
Releases
Implements [[Sample Project| iSampleProject]]
 
== Project Name ==
== Project Contributor(s) ==
[[User:dominic|Dominic Baranski]]
== Project Details == === Release 0.1 === ==== Goals ====* Add in a new build step to Buildbot* Allow Buildbot to connect with Amazon's EC2 service ==== Downloads ====     [http://matrix.senecac.on.ca/~awdelyea/EC2_build_step_nokeys.py EC2_build_step_nokeys.py]     Note: You will need to install the boto library located [httpUser://boto.googlecode.com/files/boto-0.9a.tar.gz hereArmenzg|Armen Zambrano].  ==== About ==== Basically, this first version attempts to connect to an EC2 account, and then display the connection information to show that it has connected. It uses a library called "boto", and this library is used to connect to all of [http://www.amazon.com/gp/browse.html?node=3435361 Amazon's Web Services]
To use this in a Buildbot configuration file, simply call the connect_to_ec2() function, which takes in as paramaters, an Amazon id key, and an Amazon secret key[[User:david.humphrey|Dave Humphrey]]
Example:<pre> connection = connect_to_ec2("73ghf9h38fh", "75vo87rVO*&V%O7o75v7i%i75I&5li7%i75")</pre>= Releases ==
----You can see more information about individual releases by following the links below.
=== Release * [[Buildbot and EC2/0.2 ===1]]
==== Goals ====* [[Buildbot and EC2/0.2]]
* Start New Instances* Add new instances to build slave list* Start a build on an [[Buildbot and EC2 instance build slave/0.3]]
==== Downloads ====* [[Buildbot and EC2/0.4]]
==== About ====* [[Buildbot and EC2/0.5]]
To use the * [[Buildbot and EC2 service with /0.6 | Buildbot, there needs to be some way for Buildbot to dynamically generate a list of build slavesand EC2/0. These build slaves (or just slaves) are hosted as a virtual machine on the EC2 service6/0. 7]]
This will work by extending the BuildSlave class to add in some new functionality for starting a new EC2 instance. So, every time you add a new slave to the slave list, another instance will be spwaned, so that you can spawn as many instances as you think you will need to complete a build* [[Buildbot and Testing|Tangential Testing on Buildbot Release / 0.8]]
This new BuildSlave class will start a new instance, which will then proceed to send the connection information of the master to the new instance. How exactly, I'm not exactly sure, because it takes a minute for the virtual instance to start up.  The rest basically takes care of itself. Still have to think about how to turn the new * [[Buildbot and EC2 instances off after the build is done.---- === Release /0.3 === ==== Goals ==== ==== Downloads ==== ==== About ====9]]
== Tasks ==
<table style="width: 100%;" class="standard-table" cellborder=1 border=1 cellpadding=3 cellspacing=0>
<th style="width: 1517%;"><br />'''''Task'''''<br /><br /></th> <th style="width: 4535%;"><br />'''''Details'''''<br /><br /></th> <th style="width: 105%;"><br />'''''Priority'''''<br /><br /></th> <th style="width: 10%;"><br />'''''Contributors'''''<br /><br /></th> <th style="width: 1015%;"><br />'''''Status'''''<br /><br /></th> <th style=" width: 4%;"><br />'''''Target'''''<br /><br /></th> <th style=" width: 4%;"><br />'''''Completed'''''<br /><br /></th> <th style="width: 10%;"><br />'''''Link'''''(s)<br /><br /></th>
<tr>
<td>Define new build step0.1 Release</td> <td>Add a new build step to Covering the build master config file. The build step will be in an external file and imported into the build master config file.basics</td> <td style="background-color: red; text-align: center; font-weight: bold; font-size: bigger">High</td> <td>[[User:Awdelyea|Adam Delyea]]</td>
<td style="background-color: green; text-align: center; font-weight: bold; font-size: bigger">Done</td>
<td>Adam Delyea</td>
<td>&nbsp;</td>
<td>[http:&nbsp;<//buildbot.net/repos/release/docs/buildbot.html#Writing-New-BuildSteps Writing New Build Steps]td> <td>&nbsp;</td>
</tr>
<tr>
<td>Connect To EC2 using Python0.2 Release</td> <td>Utilize the EC2 python library to connect to an account on the EC2 network. Once connected, display some information about the users account to prove it's actually connecting.</td> <td style="background-color: red; text-align: center; font-weight: bold; font-size: bigger">High</td> <td>[[User:Awdelyea|Adam Delyea]]</td>
<td style="background-color: green; text-align: center; font-weight: bold; font-size: bigger">Done</td>
<td>Adam Delyea</td>
<td>&nbsp;</td>
<td>&nbsp;<ul><li>[http://code.google.com/p/boto/ Boto Amazon Web Services Library in Python]</li><li>[http://jimmyg.org/2007/09/01/amazon-ec2-for-people-who-prefer-debian-and-python-over-fedora-and-java/ EC2 hearts Python]</litd> </ultd>&nbsp;</td>
</tr>
<tr>
<td>Start new instances0.3 Release</td> <td>Allow Buildbot to spawn an instance on demand. This could possibly be a function called right before the build slave list is populated, and would pass back an array of build slaves.</td> <td style="background-color: orangered; text-align: center; font-weight: bold; font-size: bigger">High</td> <td>[[User:Awdelyea|Adam Delyea]]</td> <td style="background-color: green; text-align: center; font-weight: bold; font-size: bigger">Done</td> <td>&nbsp;</td> <td>In Progress, On Track&nbsp;</td>
<td>&nbsp;</td>
</tr>
<tr>
<td>Add new instances to build slave list</td> <td>Allow newly created EC2 instances to be added to the build slave list0.<div style="background-color: grey; text-align: center;">After looking more closely at how the buildslave is created, this goal doesn't make any sense, since this is not really how the buildslave list works. </div> </td> <td style="background-color: grey; text-align: center; font-size: bigger">Cut</td> <td>Adam Delyea4 Release</td>
<td></td>
<td style="background-color: red; text-align: center; font-weight: bold; font-size: bigger">High</td>
<td>[[User:Awdelyea|Adam Delyea]]</td>
<td style="background-color: green; text-align: center; font-weight: bold; font-size: bigger">Done</td>
<td>14/1/08</td>
<td>14/1/08</td>
<td>&nbsp;</td>
</tr>
<tr>
<td>Start a new build on an EC2 instance0.5 Release</td> <td>Allow Buildbot to successfully start a build using a build slave on an EC2 instance</td> <td style="background-color: orangered; text-align: center; font-weight: bold; font-size: bigger">High</td> <td>[[User:Awdelyea|Adam Delyea]]</td> <tdstyle="background-color: SkyBlue; text-align: center; font-weight: bold; font-size: bigger">In Progress, On Track</td> <td>3/1/08</td> <td>&nbsp;</td>
<td>&nbsp;</td>
</tr>
<tr>
<td>Load a script on startup in LinuxTesting</td> <td>Make People needed to download the revisions and test them to make sure it possible works in different environments. Note, this will require you to have a script load when Linux startsyour own EC2 account. </td> <td style="background-color: lightbluegrey; text-align: center; font-weight: bold; font-size: bigger">LowExpired</td> <td style="background-color: gold; text-align: center; font-weight: bold; font-size: bigger">NEEDED! ***<br />[http://zenit.senecac.on.ca/wiki/index.php/User:dominic Dominic Baranski]</td> <td style="background-color: grey; text-align: center; font-weight: bold; font-size: bigger">Expired</td> <td>Not Started&nbsp;</td> <td>&nbsp;</td>
<td>&nbsp;</td>
</tr>
<tr>
<td>Testing<del>Different Builds</del></td> <td>People needed <del>Need different projects to download the revisions try and test them build, to make sure it works in different environmentsto build projects other than Mozilla</del><br />This is no longer a priority. Note, Building with Mozilla is the focus of this will require you project. The EC2 functionality can be extracted and used to have your own EC2 accountbuild any program / system. </td> <td style="background-color: lightbluegrey; text-align: center; font-weight: bold; font-size: bigger">LowExpired</td> <td style="background-color: gold; text-align: center; font-weight: bold; font-size: bigger">NEEDED! ***<br />[http://zenit.senecac.on.ca/wiki/index.php/User:dominic Dominic Baranski]</td> <td style="background-color: grey; text-align: center; font-weight: bold; font-size: bigger">Expired</td>Not Started <td>&nbsp;</td> <td>&nbsp;</td>
<td>&nbsp;</td>
</tr>
<tr>
<td>Different BuildsComplete Documentation</td> <td>Need different projects to try and build, to make sure it works to build projects other than Mozilla</td> <td style="background-color: lightblueorange; text-align: center; font-weight: bold; font-size: bigger">LowMedium</td> <td>[[User:Awdelyea|Adam Delyea]]</td> <td style="background-color: goldgreen; text-align: center; font-weight: bold; font-size: bigger">NEEDED! ***In Progress</td> <td>Not Started&nbsp;</td> <td>&nbsp;</td>
<td>&nbsp;</td>
</tr>
<br/>
<div style="background-color: gold; text-align: center; font-weight: bold;">&nbsp;<br />*** Would you like to contribute? Well here's an excellent opportunity for you to do just that! Feel free to e-mail me at <awdelyea@learngmail.senecac.on.cacom><br />&nbsp;</div><br/>
== Project Resources ==
== Project News ==
 
For a less formal, and more up-to-date view on whats happening, see the [http://www.awdelyea.wordpress.com BLOG].
2007-09-17: Project claimed!
2007-10-12: Release v0.2 loosely defined. More detail to follow.
 
2007-11-17: Release v0.2 is complete and available for download: can start an instance and start an EC2BuildSlave
 
2007-11-21: Release v0.2.1 is complete and available for download: now to include variable EC2 users, and instances, and DNS'
 
2007-12-13: Release v0.3 is complete and available for download: easier to install and setup, can use to build projects now
 
2008-01-14: Release v0.4 Defined. In the process of getting accounts and machines for Try Server Farm.
 
'''''2008-02-01: Task List and Release News is being moved to individual Release Pages. See individual Releases for more information.'''''
1
edit

Navigation menu