Add to Try Server Automated Testing Support

Introduction

We are trying to add to a Try Server, that will be set up at Seneca, automated testing support (TUnit (make check), mochitest, reftests, crashtest)

• My blog's label explaining my updates is this: (armenzg)Buildbot automated testing integration
• My code so far is in this repository: svn:Seneca Hera Try Server - automated-testing integration I will make available of the code in Hera which will have the EC2 integration and the automated testing integration

People

• Armen Zambrano G. (armenzg) #seneca channel
• People from Mozilla helping me: Rob Campbell, Robert Helmer, Ben Hearsum, Robert Sayre, Mike Shaver, Adam Delyea and Dave Humphrey

News

• 16-Apr-2008
  • Updated wiki page
  • Reftests was not running properly (it timed out - stdio - exit code -1 after 1200secs)
  • Changed DISPLAY to :2 and build 45 shows that we got GREEN with reftest
  • I have to add cronjob to run perl processchange.pl every 2 mins
  • There are 2 linux slaves on vmlinux1.hera
  • To upload a patch you must use an LDAP account (I have to investigate this)
• 15-Apr-2008
  • The try server with unit tests runs properly
  • Forked on tryserver account, sendchange.cgi and sendchange-ui.pm
  • Forked on buildmaster account, processchanges.pl and download.log
  • All of the previous files have been copied with suffix _test
  • Created on tryserver and buildmaster the folder "patches_test"
• 12-Apr-2008
  • We can now run the Unit Tests on vmlinux1.hera which is a virtual machine using the CentOS 5 image
  • We needed a DISPLAY to display in the unit tests, I set on the steps the value DISPLAY=:0
  • We needed to start an X server; I found the way to do it was startx(maybe xstart, not sure), which started the desktop environment. Used ps -ef | grep X to find out.
  • There is X :0 (started by root), Xvnc :1 and Xvnc :2
• 21-Mar-2008
  • Access to the waterfall view of Hera's buildbot and force a build by selecting lin-test link that shows you the builder's status
  • When forcing a build I do not get immediate builds
  • The tests are not running properly (Error: no display specified)
  • I do not know why do I see the code been build with gmake
  • Links to see some of the forced builds I did: #13 #14 #15 #16 #17
• 18-Mar-2008
  • Granted access to the Hera server
• 16-Feb-2008
  • There will be no release this week, since I have been working on the Buildbot and Cairo project
  • It seems that Adam has been setting up the first node and the slaves during this week. I will probably have an account next week.
• 1-Feb-2008
  • Since I could not wait for the Seneca Hera Try Server to be ready, I decided to set up buildbot on my laptop and so far is working

• 29-Jan-2008
  • SERVER NEWS: The main OS (CentOS 5.1 and Windows 2003 Server) are installed. Once they are configured we will duplicate them to the rest of the nodes.
  • Soon the server will be completely running and we will have access to it
  • I have added a repository svn:Seneca Hera Try Server - automated-testing integration an I am modifying the master.cfg file to meet the purposes; It is based on buildbot-configs/testing/unittest/
  • I might set up this Saturday my own try server at home

mozbuild.py and the test suites classes defined

• 18-Jan-2008
  • We are waiting for the Buildbot Farm to be set up at Seneca
  • At this point I have been able to run "make check" tests and mochitests. Explained in blog how to do this
  • Read Buildbot manual, discovered that I have to write some Build Steps but there's some configuration done by the community (check links section)

Releases

NOTE: I did not work on this project for a month but instead I worked on the Cairo project

Release Apr. 9th

• You can upload a patch through sendchange_test.cgi. You can also upload another .mozconfig, which overwrites default .mozconfig. Please specify the proper patch level (we suggest to create diffs from mozilla/ and select 0 as the patch level)
• Wait for a couple of minutes and refresh [1] until you see your build running
• I would like to show the steps to make everything happen in your machines but I have not setup this server. Assuming we were in the same environment/setup:

Steps

How to setup a tryserver ([2])
Setting up the buildmaster # cvs -d:pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot co -d tryserver mozilla/tools/buildbot-configs/tryserver # cd tryserver (This brings env.py, tryserver/steps.py and unittest/steps.py with others) # cvs -d:pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot co -d buildbotcustom mozilla/tools/buildbotcustom # cvs -d:pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot co -d processor mozilla/webtools/buildbot-try/processchanges.pl # mv processor/processchanges.pl . # rm -rf processor

1. TBA - IN HERE TO BE ADDED STEP TO CHECK OUT MASTER.CFG

Release Apr. 5th

• Nothing to release, I have only been able to run build with tests (with warnings) in my designated

BRANCH of slaves

• The form to upload a test to the tests branch has been created

Release Mar.21st

• Blog post about setting the automated tests on Hera - armenzg: Sess16 - Automated testing on Hera
• The files that have been modified in Hera are: master.cfg (to add the steps to do the tests), mozbuild.py(contains classes that group few steps into one and does more intelligent steps), mozconfig-linux-tests(which contains the .mozconfig for the lin-test builder) and env.py (which contains MozEnvironments['centos'] environment variables)
• Read about some errors encountered in the NEWS section in this page

0.5 release

This release was meant to just allow buildbot to run tests steps after a build. In Hera should do the same and also allow the "try-server" way.

Steps

NOTE: I assume you can build mozilla firefox. This steps are to build on Linux (my flavour is Ubuntu7.1)

# sudo apt-get install buildbot
# buildbot --version
Buildbot version: 0.7.5
Twisted version: 2.5.0
# buildbot create-master ~/BuildbotMaster
# buildbot create-slave ~/Buildslaves localhost:9989 slave1 slavepassword1
# Untar this BuildbotMasterArmen0.5.tar.gz file in your ~/BuildbotMaster directory
# buildbot start ~/BuildbotMaster
# buildbot start ~/Buildslaves/
# open your web browser and open this URL http://localhost:8010, you should see the results

NOTE 2: There is more explanation in these two posts (Sess10 and Sess11 NOTE 3: The only way for me to assure that these steps works for everyone, I would have to try it from a clean install of Ubuntu

Info

• You can see which tasks get done at the bottom of this page
• I will add the instructions on how to set up a Buildbot try server when I do it at my house
• The image on the side shows the classes that are defined in the mozbuild.py which are later on used as Buildsteps on the master.cfg file

Tasks

Task	Details	Priority	Status	Target	Link(s)
make check tests	Learn how to run Tunits (make check or xpcshell-base tests) and explained them	Med	DONE	0.4 rel.	(armenzg's post) Xpcshell based uni tests
mochitest tests	Learn how to run mochitests and explain them	Med	DONE	0.4 rel.	No post yet but have run them
Buildbot Manual	Learn basics of Buildbot that apply to this project	Med	DONE	0.4 rel.	No post - Buildbot Manual
Analize Master.cfg file	Try to see what is already written about Buildsteps relating Unit Testing	Med	Ongoing	N/A	No post - buildbot-configs/testing/unittest/
Set up Buildbot	Be able to build firefox and run unit tests	High	Done	0.5 rel.	Sess10 & Sess11
Set up "try server"	Learn how to set up my buildbot as a "try server"		TBA	?? rel.
Add steps to master.cfg	Add the steps for linux required to run tests after a build	High	Done	Mar.21st	(armenzg's post) Automated testing on Hera
Add steps for Mac and Windows	Wait until the mac and windows slaves are up and running	Med	Waiting	???
Add "try server" steps	We need to be able to "TRY" a patch and run tests in all platforms	HIGH	DONE	Apr.4
Enable windows slave		HIGH		Apr 17
Enable mac slave		HIGH		Apr 17
Sendchange form	Customize the sendchange-ui_test.pm file - It could do both things	Low		TBA
Make builds available		Med		Apr 18
Cron jobs	processchanges_test.pl and delete packaged builds	HIGH		Apr 16

Links

Description	Link
Seneca Hera Try Server	http://hera.senecac.on.ca:8010/
Buildbot configuration (1) for Unit Testing (check master.cfg file)	http://lxr.mozilla.org/mozilla/source/tools/buildbot-configs/testing/unittest/
Buildbot configuration (2) for a Tryserver (check master.cfg file)	http://lxr.mozilla.org/mozilla/source/tools/buildbot-configs/tryserver/
Building Mozilla with Buildbot	http://zenit.senecac.on.ca/wiki/index.php/Building_Mozilla_with_Buildbot