View source for Add to Try Server Automated Testing Support

== Introduction ==
We are trying to add to a Try Server, that will be set up at Seneca, automated testing support (TUnit (make check), mochitest, reftests, crashtest)

* My blog's label explaining my updates is this: [http://armenzg.blogspot.com/search/label/buildbot%20automated%20testing%20integration (armenzg)Buildbot automated testing integration]
* <strike>My code so far is in this repository: [http://svn2.assembla.com/svn/Seneca_Hera_Try_Server_Setup/automated-testing/ svn:Seneca Hera Try Server - automated-testing integration]</strike> I will make available of the code in Hera which will have the EC2 integration and the automated testing integration

== People ==
* Armen Zambrano G. (armenzg) #seneca channel
* People from Mozilla helping me: Rob Campbell, Robert Helmer, Ben Hearsum, Robert Sayre, Mike Shaver, Adam Delyea and Dave Humphrey

== News ==
* 21-Mar-2008
** Access to the [http://hera.senecac.on.ca:8010 waterfall view of Hera's buildbot] and force a build by selecting [http://hera.senecac.on.ca:8010/builders/lin-test lin-test] link that shows you the builder's status
** When forcing a build I do not get immediate builds
** The tests are not running properly ('''Error: no display specified''')
** I do not know why do I see the code been build with '''gmake'''
** Links to see some of the forced builds I did: [http://hera.senecac.on.ca:8010/builders/lin-test/builds/13 #13] [http://hera.senecac.on.ca:8010/builders/lin-test/builds/14 #14] [http://hera.senecac.on.ca:8010/builders/lin-test/builds/15 #15] [http://hera.senecac.on.ca:8010/builders/lin-test/builds/16 #16] [http://hera.senecac.on.ca:8010/builders/lin-test/builds/17 #17]
* 18-Mar-2008
** Granted access to the Hera server
* 16-Feb-2008
** There will be no release this week, since I have been working on the [[Buildbot and Cairo]] project
** It seems that Adam has been setting up the first node and the slaves during this week. I will probably have an account '''next week'''. 
* 1-Feb-2008
** Since I could not wait for the Seneca Hera Try Server to be ready, I decided to set up buildbot on my laptop and so far is working

* 29-Jan-2008
** SERVER NEWS: The main OS (CentOS 5.1 and Windows 2003 Server) are installed. Once they are configured we will duplicate them to the rest of the nodes.
** Soon the server will be completely running and we will have access to it
** I have added a repository [http://svn2.assembla.com/svn/Seneca_Hera_Try_Server_Setup/automated-testing/ svn:Seneca Hera Try Server - automated-testing integration] an I am modifying the master.cfg file to meet the purposes; It is based on [http://lxr.mozilla.org/mozilla/source/tools/buildbot-configs/testing/unittest/ buildbot-configs/testing/unittest/]
** I might set up this Saturday my own try server at home
[[Image:Browsing mozbuild(py).jpeg|thumb|mozbuild.py and the test suites classes defined]]
* 18-Jan-2008
** We are waiting for the Buildbot Farm to be set up at Seneca
** At this point I have been able to run "make check" tests and mochitests. Explained in blog how to do this
** Read Buildbot manual, discovered that I have to write some Build Steps but there's some configuration done by the community (check links section)

== Releases ==
'''NOTE: I did not work on this project for a month but instead I worked on the Cairo project'''
<blockquote>
=== Release Mar.21st ===
* Blog post about setting the automated tests on Hera - [http://armenzg.blogspot.com/2008/03/armenzg-sess16-automated-testing-on.html armenzg: Sess16 - Automated testing on Hera]
* The files that have been modified in Hera are: master.cfg (to add the steps to do the tests), mozbuild.py(contains classes that group few steps into one and does more intelligent steps), mozconfig-linux-tests(which contains the .mozconfig for the lin-test builder) and env.py (which contains MozEnvironments['centos'] environment variables)
* Read about some errors encountered in the NEWS section in this page
=== 0.5 release ===
This release was meant to just allow buildbot to run tests steps after a build. In Hera should do the same and also allow the "try-server" way.
==== Steps ====
NOTE: I assume you can build mozilla firefox. This steps are to build on Linux (my flavour is Ubuntu7.1)
 # sudo apt-get install buildbot
 # buildbot --version
 Buildbot version: 0.7.5
 Twisted version: 2.5.0
 # buildbot create-master ~/BuildbotMaster
 # buildbot create-slave ~/Buildslaves localhost:9989 slave1 slavepassword1
 # Untar this [http://matrix.senecac.on.ca/~azambran/mozilla/buildbot-config/BuildbotMasterArmen0.5.tar.gz BuildbotMasterArmen0.5.tar.gz file] in your ~/BuildbotMaster directory
 # buildbot start ~/BuildbotMaster
 # buildbot start ~/Buildslaves/
 # open your web browser and open this URL http://localhost:8010, you should see the results
NOTE 2: There is more explanation in these two posts ([http://armenzg.blogspot.com/2008/02/armenzg-sess10-buildbot-initial-set-up.html Sess10] and [http://armenzg.blogspot.com/2008/02/armenzg-sess11-how-to-set-up-buildbot.html Sess11]
NOTE 3: The only way for me to assure that these steps works for everyone, I would have to try it from a clean install of Ubuntu

</blockquote>
=== Info ===
* You can see which tasks get done at the bottom of this page
* <strike>I will add the instructions on how to set up a Buildbot try server when I do it at my house</strike>
* The image on the side shows the classes that are defined in the [http://svn2.assembla.com/svn/Seneca_Hera_Try_Server_Setup/automated-testing/mozbuild.py mozbuild.py] which are later on used as Buildsteps on the [http://svn2.assembla.com/svn/Seneca_Hera_Try_Server_Setup/automated-testing/master.cfg master.cfg] file

== Tasks ==
<table style="width: 100%;" class="standard-table" cellborder=1 border=1 cellpadding=3 cellspacing=0>
    <th style="width: 15%;">Task</th>
    <th style="width: 42%>Details</th>
    <th style="width: 5%;">Priority</th>
    <th style="width: 5%;">Status</th>
    <th style="width: 8%;">Target</th>
    <th style="width: 25%;">Link(s)</th>

    <tr>
        <td>'''make check''' tests</td>
        <td>Learn how to run Tunits (make check or xpcshell-base tests) and explained them</td>
        <td style="background-color: Orange; text-align: center; font-weight: bold; font-size: bigger">Med</td>
        <td style="background-color: Green; text-align: center; font-weight: bold;">DONE</td>
        <td style="background-color: YellowGreen; text-align: center; font-weight: bold;">0.4 rel.</td>
        <td>[http://armenzg.blogspot.com/2008/01/sess5-xpcshell-based-unit-tests-tunits.html (armenzg's post) Xpcshell based uni tests]</td>
    </tr>
    <tr>
        <td>'''mochitest''' tests</td>
        <td>Learn how to run mochitests and explain them</td>
        <td style="background-color: Orange; text-align: center; font-weight: bold; font-size: bigger">Med</td>
        <td style="background-color: Green; text-align: center; font-weight: bold;">DONE</td>
        <td style="background-color: YellowGreen; text-align: center; font-weight: bold;">0.4 rel.</td>
        <td>No post yet but have run them</td>
    </tr>
    <tr>
        <td>Buildbot Manual</td>
        <td>Learn basics of Buildbot that apply to this project</td>
        <td style="background-color: Orange; text-align: center; font-weight: bold; font-size: bigger">Med</td>
        <td style="background-color: Green; text-align: center; font-weight: bold;">DONE</td>
        <td style="background-color: YellowGreen; text-align: center; font-weight: bold;">0.4 rel.</td>
        <td>No post - [http://buildbot.sourceforge.net/manual-0.7.5.html Buildbot Manual]</td>
    </tr>
    <tr>
        <td>Analize Master.cfg file</td>
        <td>Try to see what is already written about Buildsteps relating Unit Testing</td>
        <td style="background-color: Orange; text-align: center; font-weight: bold; font-size: bigger">Med</td>
        <td style="background-color: Green; text-align: center; font-weight: bold;">Ongoing</td>
        <td style="background-color: YellowGreen; text-align: center; font-weight: bold;">N/A</td>
        <td>No post - [http://lxr.mozilla.org/mozilla/source/tools/buildbot-configs/testing/unittest/ buildbot-configs/testing/unittest/]</td>
    </tr>
    <tr>
        <td>Set up Buildbot</td><!-- TASK -->
        <td>Be able to build firefox and run unit tests</td><!-- DETAILS -->
        <td style="background-color: Red; text-align: center; font-weight: bold; font-size: bigger">High</td><!-- Priority-->
        <td style="background-color: Green; text-align: center; font-weight: bold;">Done</td><!-- Status-->
        <td style="background-color: YellowGreen; text-align: center; font-weight: bold;">0.5 rel.</td><!-- For which release-->
        <td>[http://armenzg.blogspot.com/2008/02/armenzg-sess10-buildbot-initial-set-up.html Sess10] & [http://armenzg.blogspot.com/2008/02/armenzg-sess11-how-to-set-up-buildbot.html Sess11]</td><!-- Link to blog post-->
    </tr>
    <tr>
        <td><strike>Set up "try server"</strike></td><!-- TASK -->
        <td><strike>Learn how to set up my buildbot as a "try server"</strike></td><!-- DETAILS -->
        <td style="background-color: Orange; text-align: center; font-weight: bold; font-size: bigger">&nbsp;</td><!-- Priority-->
        <td style="background-color: White; text-align: center; font-weight: bold;">TBA</td><!-- Status-->
        <td style="background-color: YellowGreen; text-align: center; font-weight: bold;">?? rel.</td><!-- For which release-->
        <td>&nbsp;</td><!-- Link to blog post-->
    </tr>
    <tr>
<!-- TASK -->    <td>Add steps to master.cfg</td>
<!-- DETAILS --> <td>Add the steps for linux required to run tests after a build</td>
<!-- Priority--> <td style="background-color: Orange; text-align: center; font-weight: bold; font-size: bigger">High</td>
<!-- Status-->   <td style="background-color: Green; text-align: center; font-weight: bold;">Done</td>
<!-- For when--> <td style="background-color: YellowGreen; text-align: center; font-weight: bold;">Mar.21st</td>
<!-- Links-->    <td>[http://armenzg.blogspot.com/2008/03/armenzg-sess16-automated-testing-on.html (armenzg's post) Automated testing on Hera]</td>
    </tr>
    <tr>
<!-- TASK -->    <td>Add steps for Mac and Windows</td>
<!-- DETAILS --> <td>Wait until the mac and windows slaves are up and running</td>
<!-- Priority--> <td style="background-color: Orange; text-align: center; font-weight: bold; font-size: bigger">Med</td>
<!-- Status-->   <td style="background-color: White; text-align: center; font-weight: bold;">Waiting</td>
<!-- For when--> <td style="background-color: YellowGreen; text-align: center; font-weight: bold;">???</td>
<!-- Links-->    <td>&nbsp;</td>
    </tr>
    <tr>
<!-- TASK -->    <td>Add "try server" steps</td>
<!-- DETAILS --> <td>We need to be able to "TRY" a patch and run tests in all platforms</td>
<!-- Priority--> <td style="background-color: Orange; text-align: center; font-weight: bold; font-size: bigger">HIGH</td>
<!-- Status-->   <td style="background-color: White; text-align: center; font-weight: bold;">&nbsp;</td>
<!-- For when--> <td style="background-color: YellowGreen; text-align: center; font-weight: bold;">Apr.4</td>
<!-- Links-->    <td>&nbsp;</td>
    </tr>
    <tr>
<!-- TASK -->    <td>&nbsp;</td>
<!-- DETAILS --> <td>&nbsp;</td>
<!-- Priority--> <td style="background-color: Orange; text-align: center; font-weight: bold; font-size: bigger">&nbsp;</td>
<!-- Status-->   <td style="background-color: White; text-align: center; font-weight: bold;">&nbsp;</td>
<!-- For when--> <td style="background-color: YellowGreen; text-align: center; font-weight: bold;">&nbsp;</td>
<!-- Links-->    <td>&nbsp;</td>
    </tr>
<table>

== Links ==
{| border="1"
|-
|class="header"|Description
|class="header"|Link
|-
| Seneca Hera Try Server
| http://hera.senecac.on.ca:8010/
|-
| Buildbot configuration (1) for Unit Testing (check master.cfg file)
| http://lxr.mozilla.org/mozilla/source/tools/buildbot-configs/testing/unittest/
|-
| Buildbot configuration (2) for a Tryserver (check master.cfg file)
| http://lxr.mozilla.org/mozilla/source/tools/buildbot-configs/tryserver/
|-
|
|
|-
|
|

|-
| Building Mozilla with Buildbot
| http://zenit.senecac.on.ca/wiki/index.php/Building_Mozilla_with_Buildbot
|-
|}