Difference between revisions of "Buildbot and EC2"

From CDOT Wiki
Jump to: navigation, search
(→‎Tasks)
(→‎Releases)
 
(85 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
Implements [[Sample Project| iSampleProject]]
 +
 
== Project Name ==
 
== Project Name ==
  
Line 14: Line 16:
  
 
== Project Contributor(s) ==
 
== Project Contributor(s) ==
 +
[[User:dominic|Dominic Baranski]]
  
−
== Project Details ==
+
[[User:Armenzg|Armen Zambrano]]
  
−
===      Release 0.1 ===
+
[[User:david.humphrey|Dave Humphrey]]
−
'''Goals:'''
 
−
* Add in a new build step to Buildbot
 
−
* Allow Buildbot to connect with Amazon's EC2 service
 
  
 +
== Releases ==
  
−
Basically, the first thing that needs to be done is add a new step to Buildbot's build process. Once a new step has been added, the next step is to make this new build step access my EC2 account.  
+
You can see more information about individual releases by following the links below.
  
−
After I can actually access my EC2 account through Buildbot, then a possible next step would be to make Buildbot check to see if I have any open Virtual Instances running already (no point in starting another instance if ones running already).
+
* [[Buildbot and EC2/0.1]]
  
−
One thing that may need to be kept in mind is that Amazon charges 10 cents per hour, or part of an hour, per instance. So, each time you start a new instance, it costs 10 cents. The best way I can think of doing this is make each instance you start, last for at least an hour, so that if you start another build in 10 mins, you can continue to use the same instance instead of starting a new one.  
+
* [[Buildbot and EC2/0.2]]
  
−
For this, Buildbot would need to run a service on the OS to constantly check to see if an instance is approaching the 1 hour mark, and not running any build slaves. Not quite sure how I'd do that, but it's something to keep in mind.
+
* [[Buildbot and EC2/0.3]]
−
===      Release 0.2 ===
+
 
−
===      Release 0.3 ===
+
* [[Buildbot and EC2/0.4]]
 +
 
 +
* [[Buildbot and EC2/0.5]]
 +
 
 +
* [[Buildbot and EC2/0.6 | Buildbot and EC2/0.6/0.7]]
 +
 
 +
* [[Buildbot and Testing|Tangential Testing on Buildbot Release / 0.8]]
 +
 
 +
* [[Buildbot and EC2/0.9]]
  
 
== Tasks ==
 
== Tasks ==
  
−
<table style="width: 100%;" class="standard-table" cellborder=1 border=1 cellpadding=2 cellspacing=0>
+
<table style="width: 100%;" class="standard-table" cellborder=1 border=1 cellpadding=3 cellspacing=0>
−
     <th style="width: 15%">Task</th>
+
     <th style=" width: 17%;"><br />'''''Task'''''<br /><br /></th>
−
     <th>Details</th>
+
     <th style=" width: 35%;"><br />'''''Details'''''<br /><br /></th>
−
     <th style="width: 10%">Priority</th>
+
     <th style=" width: 5%;"><br />'''''Priority'''''<br /><br /></th>
−
     <th style="width: 15%">Contributors</th>
+
     <th style=" width: 10%;"><br />'''''Contributors'''''<br /><br /></th>
−
     <th style="width: 15%">Status</th>
+
     <th style=" width: 15%;"><br />'''''Status'''''<br /><br /></th>
−
     <th style="width: 15%">Link(s)</th>
+
     <th style=" width: 4%;"><br />'''''Target'''''<br /><br /></th>
 +
    <th style=" width: 4%;"><br />'''''Completed'''''<br /><br /></th>
 +
    <th style=" width: 10%;"><br />'''''Link'''''(s)<br /><br /></th>
 +
 
 +
    <tr>
 +
        <td>0.1 Release</td>
 +
        <td>Covering the basics</td>
 +
        <td  style="background-color: red; text-align: center; font-weight: bold; font-size: bigger">High</td>
 +
        <td>[[User:Awdelyea|Adam Delyea]]</td>
 +
        <td style="background-color: green; text-align: center; font-weight: bold; font-size: bigger">Done</td>
 +
        <td>&nbsp;</td>
 +
        <td>&nbsp;</td>
 +
        <td>&nbsp;</td>
 +
    </tr>
 +
 
 +
    <tr>
 +
        <td>0.2 Release</td>
 +
        <td></td>
 +
        <td  style="background-color: red; text-align: center; font-weight: bold; font-size: bigger">High</td>
 +
        <td>[[User:Awdelyea|Adam Delyea]]</td>
 +
        <td style="background-color: green; text-align: center; font-weight: bold; font-size: bigger">Done</td>
 +
        <td>&nbsp;</td>
 +
        <td>&nbsp;</td>
 +
        <td>&nbsp;</td>
 +
    </tr>
 +
 
 +
    <tr>
 +
        <td>0.3 Release</td>
 +
        <td></td>
 +
        <td  style="background-color: red; text-align: center; font-weight: bold; font-size: bigger">High</td>
 +
        <td>[[User:Awdelyea|Adam Delyea]]</td>
 +
        <td style="background-color: green; text-align: center; font-weight: bold; font-size: bigger">Done</td>
 +
        <td>&nbsp;</td>
 +
        <td>&nbsp;</td>
 +
        <td>&nbsp;</td>
 +
    </tr>
 +
 
 +
    <tr>
 +
        <td>0.4 Release</td>
 +
        <td></td>
 +
        <td  style="background-color: red; text-align: center; font-weight: bold; font-size: bigger">High</td>
 +
        <td>[[User:Awdelyea|Adam Delyea]]</td>
 +
        <td style="background-color: green; text-align: center; font-weight: bold; font-size: bigger">Done</td>
 +
        <td>14/1/08</td>
 +
        <td>14/1/08</td>
 +
        <td>&nbsp;</td>
 +
    </tr>
 +
 
 +
    <tr>
 +
        <td>0.5 Release</td>
 +
        <td></td>
 +
        <td  style="background-color: red; text-align: center; font-weight: bold; font-size: bigger">High</td>
 +
        <td>[[User:Awdelyea|Adam Delyea]]</td>
 +
        <td style="background-color: SkyBlue; text-align: center; font-weight: bold; font-size: bigger">In Progress</td>
 +
        <td>3/1/08</td>
 +
        <td>&nbsp;</td>
 +
        <td>&nbsp;</td>
 +
    </tr>
  
 
     <tr>
 
     <tr>
−
         <td>Define new build step</td>
+
         <td>Testing</td>
−
         <td>Add a new build step to the build master config file. The build step will be in an external file and imported into the build master config file.</td>
+
         <td>People needed to download the revisions and test them to make sure it works in different environments. Note, this will require you to have your own EC2 account. </td>
−
         <td style="background-color: red; text-align: center; font-weight: bold; font-size: bigger">High</td>
+
         <td style="background-color: grey; text-align: center; font-weight: bold; font-size: bigger">Expired</td>
−
         <td>Adam Delyea</td>
+
         <td style="background-color: gold; text-align: center; font-weight: bold; font-size: bigger">NEEDED! ***<br />[http://zenit.senecac.on.ca/wiki/index.php/User:dominic Dominic Baranski]</td>
−
         <td>In Progress, On Track</td>
+
         <td style="background-color: grey; text-align: center; font-weight: bold; font-size: bigger">Expired</td>
−
         <td>[http://buildbot.net/repos/release/docs/buildbot.html#Writing-New-BuildSteps Writing New Build Steps]</td>
+
         <td>&nbsp;</td>
 +
        <td>&nbsp;</td>
 +
        <td>&nbsp;</td>
 
     </tr>
 
     </tr>
  
 
     <tr>
 
     <tr>
−
         <td>Connect To EC2 using Python</td>
+
         <td><del>Different Builds</del></td>
−
         <td>Utilize the EC2 python library to connect to an account on the EC2 network. Once connected, display some information about the users account to prove it's actually connecting.</td>
+
         <td><del>Need different projects to try and build, to make sure it works to build projects other than Mozilla</del><br />This is no longer a priority. Building with Mozilla is the focus of this project. The EC2 functionality can be extracted and used to build any program / system.</td>
−
         <td style="background-color: red; text-align: center; font-weight: bold; font-size: bigger">High</td>
+
         <td style="background-color: grey; text-align: center; font-weight: bold; font-size: bigger">Expired</td>
−
         <td>Adam Delyea</td>
+
         <td style="background-color: gold; text-align: center; font-weight: bold; font-size: bigger">NEEDED! ***<br />[http://zenit.senecac.on.ca/wiki/index.php/User:dominic Dominic Baranski]</td>
−
         <td>In Progress, On Track</td>
+
         <td style="background-color: grey; text-align: center; font-weight: bold; font-size: bigger">Expired</td>
−
         <td><ul><li>[http://code.google.com/p/boto/ Boto Amazon Web Services Library in Python]</li><li>[http://jimmyg.org/2007/09/01/amazon-ec2-for-people-who-prefer-debian-and-python-over-fedora-and-java/ Amazon-ec2-for-people-who-prefer-debian-and-python-over-fedora-and-java]</li></ul></td>
+
         <td>&nbsp;</td>
 +
        <td>&nbsp;</td>
 +
        <td>&nbsp;</td>
 +
    </tr>
 +
 
 +
    <tr>
 +
        <td>Complete Documentation</td>
 +
        <td></td>
 +
        <td style="background-color: orange; text-align: center; font-weight: bold; font-size: bigger">Medium</td>
 +
        <td>[[User:Awdelyea|Adam Delyea]]</td>
 +
        <td style="background-color: green; text-align: center; font-weight: bold; font-size: bigger">In Progress</td>
 +
        <td>&nbsp;</td>
 +
        <td>&nbsp;</td>
 +
        <td>&nbsp;</td>
 
     </tr>
 
     </tr>
 
</table>
 
</table>
 +
 +
== Contributions ==
 +
 +
<br/>
 +
<div style="background-color: gold; text-align: center;font-weight: bold;">&nbsp;<br />*** Would you like to contribute? Well here's an excellent opportunity for you to do just that! Feel free to e-mail me at <awdelyea@gmail.com><br />&nbsp;</div><br/>
  
 
== Project Resources ==
 
== Project Resources ==
Line 69: Line 155:
  
 
== Project News ==
 
== Project News ==
 +
 +
For a less formal, and more up-to-date view on whats happening, see the [http://www.awdelyea.wordpress.com BLOG].
  
 
2007-09-17: Project claimed!
 
2007-09-17: Project claimed!
  
 
2007-10-11: Found a python EC2 library that has all of the functions I will need for interacting with EC2. Just need to write a custom build step to run these library functions.
 
2007-10-11: Found a python EC2 library that has all of the functions I will need for interacting with EC2. Just need to write a custom build step to run these library functions.
 +
 +
2007-10-12: Release v0.1 is complete and available for download.
 +
 +
2007-10-12: Release v0.2 loosely defined. More detail to follow.
 +
 +
2007-11-17: Release v0.2 is complete and available for download: can start an instance and start an EC2BuildSlave
 +
 +
2007-11-21: Release v0.2.1 is complete and available for download: now to include variable EC2 users, and instances, and DNS'
 +
 +
2007-12-13: Release v0.3 is complete and available for download: easier to install and setup, can use to build projects now
 +
 +
2008-01-14: Release v0.4 Defined. In the process of getting accounts and machines for Try Server Farm.
 +
 +
'''''2008-02-01: Task List and Release News is being moved to individual Release Pages. See individual Releases for more information.'''''

Latest revision as of 10:23, 26 March 2008

Implements iSampleProject

Project Name

Buildbot and EC2

Project Description

The BuildBot is software for automatically building and testing other software. EC2 is Amazon's "Elastic Compute Cloud", a service that lets you run Virtual Machines on Amazon's computing grid using a web API. It would be awesome if BuildBot had an EC2 module, and could spawn Virtual Machines on-demand to perform tasks. Conveniently, BuildBot is written in Python, and there's a Python library for interfacing with EC2.

References: Amazon EC2 Basics For Python Programmers

Project Leader(s)

Adam Delyea

Project Contributor(s)

Dominic Baranski

Armen Zambrano

Dave Humphrey

Releases

You can see more information about individual releases by following the links below.

Tasks


Task
Details
Priority
Contributors
Status
Target
Completed
Link(s)
0.1 Release Covering the basics High Adam Delyea Done      
0.2 Release High Adam Delyea Done      
0.3 Release High Adam Delyea Done      
0.4 Release High Adam Delyea Done 14/1/08 14/1/08  
0.5 Release High Adam Delyea In Progress 3/1/08    
Testing People needed to download the revisions and test them to make sure it works in different environments. Note, this will require you to have your own EC2 account. Expired NEEDED! ***
Dominic Baranski		 Expired      
Different Builds Need different projects to try and build, to make sure it works to build projects other than Mozilla
This is no longer a priority. Building with Mozilla is the focus of this project. The EC2 functionality can be extracted and used to build any program / system.		 Expired NEEDED! ***
Dominic Baranski		 Expired      
Complete Documentation Medium Adam Delyea In Progress      

Contributions


 
*** Would you like to contribute? Well here's an excellent opportunity for you to do just that! Feel free to e-mail me at <awdelyea@gmail.com>
 

Project Resources

Project News

For a less formal, and more up-to-date view on whats happening, see the BLOG.

2007-09-17: Project claimed!

2007-10-11: Found a python EC2 library that has all of the functions I will need for interacting with EC2. Just need to write a custom build step to run these library functions.

2007-10-12: Release v0.1 is complete and available for download.

2007-10-12: Release v0.2 loosely defined. More detail to follow.

2007-11-17: Release v0.2 is complete and available for download: can start an instance and start an EC2BuildSlave

2007-11-21: Release v0.2.1 is complete and available for download: now to include variable EC2 users, and instances, and DNS'

2007-12-13: Release v0.3 is complete and available for download: easier to install and setup, can use to build projects now

2008-01-14: Release v0.4 Defined. In the process of getting accounts and machines for Try Server Farm.

2008-02-01: Task List and Release News is being moved to individual Release Pages. See individual Releases for more information.

Retrieved from "https://wiki.cdot.senecapolytechnic.ca/w/index.php?title=Buildbot_and_EC2&oldid=15292"