Add to Try Server Automated Testing Support

Introduction

We are trying to add to a Try Server, that will be set up at Seneca, automated testing support (TUnit (make check), mochitest, reftests, crashtest)

• My blog's label explaining my updates is this: (armenzg)Buildbot automated testing integration
• My code so far is in this repository: svn:Seneca Hera Try Server - automated-testing integration I will make available of the code in Hera which will have the EC2 integration and the automated testing integration

People

• Armen Zambrano G. (armenzg) #seneca channel
• People from Mozilla helping me: Rob Campbell, Robert Helmer, Ben Hearsum, Robert Sayre, Mike Shaver, Adam Delyea and Dave Humphrey

News

• 21-Mar-2008
  • Access to the waterfall view of Hera's buildbot and force a build by selecting lin-test link that shows you the builder's status
  • When forcing a build I do not get immediate builds
  • The tests are not running properly (Error: no display specified)
  • I do not know why do I see the code been build with gmake
  • Links to see some of the forced builds I did: #13 #14 #15 #16 #17
• 18-Mar-2008
  • Granted access to the Hera server
• 16-Feb-2008
  • There will be no release this week, since I have been working on the Buildbot and Cairo project
  • It seems that Adam has been setting up the first node and the slaves during this week. I will probably have an account next week.
• 1-Feb-2008
  • Since I could not wait for the Seneca Hera Try Server to be ready, I decided to set up buildbot on my laptop and so far is working

• 29-Jan-2008
  • SERVER NEWS: The main OS (CentOS 5.1 and Windows 2003 Server) are installed. Once they are configured we will duplicate them to the rest of the nodes.
  • Soon the server will be completely running and we will have access to it
  • I have added a repository svn:Seneca Hera Try Server - automated-testing integration an I am modifying the master.cfg file to meet the purposes; It is based on buildbot-configs/testing/unittest/
  • I might set up this Saturday my own try server at home

mozbuild.py and the test suites classes defined

• 18-Jan-2008
  • We are waiting for the Buildbot Farm to be set up at Seneca
  • At this point I have been able to run "make check" tests and mochitests. Explained in blog how to do this
  • Read Buildbot manual, discovered that I have to write some Build Steps but there's some configuration done by the community (check links section)

Releases

NOTE: I did not work on this project for a month but instead I worked on the Cairo project

Release Mar.21st

• Blog post about setting the automated tests on Hera - armenzg: Sess16 - Automated testing on Hera
• The files that have been modified in Hera are: master.cfg (to add the steps to do the tests), mozbuild.py(contains classes that group few steps into one and does more intelligent steps), mozconfig-linux-tests(which contains the .mozconfig for the lin-test builder) and env.py (which contains MozEnvironments['centos'] environment variables)
• Read about some errors encountered in the NEWS section in this page

0.5 release

This release was meant to just allow buildbot to run tests steps after a build. In Hera should do the same and also allow the "try-server" way.

Steps

NOTE: I assume you can build mozilla firefox. This steps are to build on Linux (my flavour is Ubuntu7.1) # sudo apt-get install buildbot # buildbot --version Buildbot version: 0.7.5 Twisted version: 2.5.0 # buildbot create-master ~/BuildbotMaster # buildbot create-slave ~/Buildslaves localhost:9989 slave1 slavepassword1 # Untar this BuildbotMasterArmen0.5.tar.gz file in your ~/BuildbotMaster directory # buildbot start ~/BuildbotMaster # buildbot start ~/Buildslaves/ # open your web browser and open this URL http://localhost:8010, you should see the results NOTE 2: There is more explanation in these two posts (Sess10 and Sess11 NOTE 3: The only way for me to assure that these steps works for everyone, I would have to try it from a clean install of Ubuntu

Info

• You can see which tasks get done at the bottom of this page
• I will add the instructions on how to set up a Buildbot try server when I do it at my house
• The image on the side shows the classes that are defined in the mozbuild.py which are later on used as Buildsteps on the master.cfg file

Tasks

Task	Details	Priority	Status	Target	Link(s)
make check tests	Learn how to run Tunits (make check or xpcshell-base tests) and explained them	Med	DONE	0.4 rel.	(armenzg's post) Xpcshell based uni tests
mochitest tests	Learn how to run mochitests and explain them	Med	DONE	0.4 rel.	No post yet but have run them
Buildbot Manual	Learn basics of Buildbot that apply to this project	Med	DONE	0.4 rel.	No post - Buildbot Manual
Analize Master.cfg file	Try to see what is already written about Buildsteps relating Unit Testing	Med	Ongoing	N/A	No post - buildbot-configs/testing/unittest/
Set up Buildbot	Be able to build firefox and run unit tests	High	Done	0.5 rel.	Sess10 & Sess11
Set up "try server"</strike</td>	Learn how to set up my buildbot as a "try server"</td>	</td>	TBA</td>	?? rel.</td>	</td> </tr>
Add steps to master.cfg</td>	Add the steps for linux required to run tests after a build</td>	High</td>	TBA</td>	0.x rel.</td>	armenzg: Sess16 - Automated testing on Hera</td> </tr>
Add steps for Mac and Windows</td>	Wait until the mac and windows slaves are up and running</td>	Med</td>	Waiting</td>	???</td>	</td> </tr>
Add "try server" steps</td>	We need to be able to "TRY" a patch and run tests in all platforms</td>	HIGH</td>	</td>	Apr.4</td>	</td> </tr>
</td>	</td>	</td>	</td>	</td>	</td> </tr> Links Description Link Seneca Hera Try Server http://hera.senecac.on.ca:8010/ Buildbot configuration (1) for Unit Testing (check master.cfg file) http://lxr.mozilla.org/mozilla/source/tools/buildbot-configs/testing/unittest/ Buildbot configuration (2) for a Tryserver (check master.cfg file) http://lxr.mozilla.org/mozilla/source/tools/buildbot-configs/tryserver/ Building Mozilla with Buildbot http://zenit.senecac.on.ca/wiki/index.php/Building_Mozilla_with_Buildbot