== Project Contributor(s) ==
[[User:dominic|Dominic Baranski]]
== Project Details ==
=== Release 0.1 === ==== Goals ====* Add in a new build step to [[Buildbot* Allow Buildbot to connect with Amazon's and EC2 service ==== Downloads ==== [http://matrix.senecac.on.ca/~awdelyea/EC2_build_step_nokeys.py EC2_build_step_nokeys.py] Note: You will need to install the boto library located [http://boto.googlecode.com/files/boto-0.9a.tar.gz here1]. ==== About ==== Basically, this first version attempts to connect to an EC2 account, and then display the connection information to show that it has connected. It uses a library called "boto", and this library is used to connect to all of [http://www.amazon.com/gp/browse.html?node=3435361 Amazon's Web Services] To use this in a Buildbot configuration file, simply call the connect_to_ec2() function, which takes in as paramaters, an Amazon id key, and an Amazon secret key. Example:<pre> connection = connect_to_ec2("73ghf9h38fh", "75vo87rVO*&V%O7o75v7i%i75I&5li7%i75")</pre> ----
=== Release 0.2 ===
==== Goals ==== * Start New Instances* Add new instances to build slave list* Start a build on an EC2 instance build slave ==== Downloads ==== [http://zenit.senecac.on.ca/wiki/imgs/EC2BuildSlave_nokey.zip EC2BuildSlave.py] *Just put this file into your buildbot installation folder so python can access it<br/> [http://zenit.senecac.on.ca/wiki/imgs/Master.zip Master.cfg] ==== About ==== To use the EC2 service with Buildbot, there needs to be some way for Buildbot to dynamically generate a list of build slaves. These build slaves (or just slaves) are hosted as a virtual machine on the and EC2 service. This will work by extending the BuildSlave class to add in some new functionality for starting a new EC2 instance. So, every time you add a new slave to the slave list, another instance will be spwaned, so that you can spawn as many instances as you think you will need to complete a build. This new BuildSlave class will start a new instance, which will then proceed to send the connection information of the master to the new instance. How exactly, I'm not exactly sure, because it takes a minute for the virtual instance to start up. The rest basically takes care of itself. Still have to think about how to turn the new EC2 instances off after the build is done. ====/0.2.1=========About=====I have made a slight modification to the EC2BuildSlave. It now requires some additional input, because I had been hardcoding some values in, and that's sloppy, so I corrected it to allow you to put any variable parameters in the config file. Basically, you will lay out a new EC2BuildSlave like this: from buildbot.EC2BuildSlave import EC2BuildSlave c['slaves'] = [EC2BuildSlave("bot1name", "bot1passwd", "public key", "private key", "dns", "instance-id")] //Where dns = www.whatever.com:8898, or 88.123.55.456:8898 //and public/private key is your public/private key from Amazon //and instance-id is the instance you would like to launch ("ami-87d530ee") =====Downloads=====* [[http://zenit.senecac.on.ca/wiki/imgs/Buildbot_EC2.zip Patch]] The EC2BuildSlave.py goes into your python's framework folder for Buildbot (note: on my mac, this is<br/> /Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/buildbot/) ----
=== Release 0.3 ===
[[Buildbot and EC2/0.3]]
==== Goals ====
* Build Mozilla using an EC2 Instance
* Complete Documentation
* Generic EC2 instance for Buildbot Slaves
* Startup Scripts for making your own EC2 instances
==== Downloads ====
* [http://zenit.senecac.on.ca/wiki/imgs/EC2.zip EC2.zip]
==== About ====
So this iteration is basically about getting this project into the “publically useable” stage. Right now, it’s fairly easy to set up these modifications to use EC2 instances in any Buildbot build project.
To use, you need to have Buildbot installed already. Then, you need to download the BOTO python library from Google, and place the boto folder from that zip file into your build-master directory. Then, you need to extract this patch into your build-master directory. Thats it for system prep.
Then, you need to go to AWS.AMAZON.COM and sign up for an account. When you do that, you will need to get your Public Key, and Secret Key to paste into your build-master’s master.cfg file.
In your build-masters master.cfg file, where you would normally specify your list of BuildSlaves, you will be replacing BuildSlave with EC2BuildSlave. This custom class is what allows Buildbot to start EC2 instances and create slaves to use on them. You need to specify a few extra variables in the slave configuration.
By default, you need to specify:
* Bot Name
* Bot Password
* Public Key
* Private Key
* [IP Address of Build-Master:Port] [Bot Name] [Bot Password]
By supplying the build-master’s IP address to the EC2 slave, you are allowing the build-slave on the EC2 instance to be pointed at your machine. If you have a DNS, then that will work in this place as well.
There is a generic Amazon EC2 instance (also called an AMI) created that is set by default in the EC2BuildSlave class. However, if you desire, you may also insert a custom AMI instance ID to the EC2BuildSlave and have it start up your own AMI. See the custom startup script, in the 0.3 deliverable zip, to insert into your instance to have your build-slave created at run-time.
----
=== Release 0.4 ===
==== Goals ====*Run a Try server @ Home*Make Try server use EC2 ==== Downloads ==== None, but when this is completed, the zipped archive of the try server will go here ==== About ==== For this release, it was important to make this project more useful for Mozilla. <br /><br />For EC2 to be useful, you have to use it when you have periods of inactivity, or periods of sudden increased activity. This appears to happen most often on Mozilla's Try Server. <br /><br /> The Try Server lets developers submit a patch to the trunk, [[Buildbot and have it build on all 3 OS's, without submitting their change to trunk. So, it's just a place where developers can try and see if their patch will work on all 3 major OS's. <br /><br /> Since developers are using the Try Server continuously, and it's a service that cannot really be taken offline, Seneca will be hosting it's own Try Server. Right now it is very unclear how many machines will be made available to this Try Server, but there should be about 10 machines available for this project at the beginning. As the project continues, more servers will likely be added.<br /><br /> Since time is an issue in getting access to the servers @ Seneca, the first little bit in getting the Try Server to use EC2 build slaves will be running on an external (from Seneca) server. Once that is completed, a zip of the buildbot files will be made available here. ===== Server Location ===== [http://CitadelStudios0.ca Try Server Patch Submission Page1]]
== Tasks ==