Hera Try Server Administration

Try Server (Front node)

• To connect to the tryserver do it through ssh to hera.senecac.on.ca
• The folders "cgi-bin", "html" and "patches" are aliased to show to the public

Master (Front node)

• The basedir of the buildbot is the home directory, therefore "buildobt restart ."
• Run "perl process_changes_test.pl" to read the patches from the "tryserver" and notify of changes to the master. Every 2 minutes a cronjob takes care of running this command. Patches get copied to "patches" or "patches_test" (this may be unified later on)

Slaves

NOTE: For any account credentials talk with Dave Humphrey
NOTE2: The Mac slave has the slave in /opt/slave and it is owned by armenzg account, shall we create a "buildslave" account to own it?

Connecting

• LINUX
  • You have to connect first to the front node and the "ssh buildslave@vmlinux1"
  • You can also connect with VMware to hera.senecac.on.ca:1904
• MAC
  • You connect directly with ssh to a specific IP (to be added) and with your individual account
• WINDOWS
  • Use "Remote Desktop Connection" (comes with Windows) to connect to hera.senecac.on.ca:31389 with buildslave account

Creation

NOTE: Make sure that the tryserver has in ~/.ssh/authorized_keys the keys of the slaves in it AND make sure that you ssh at least once to the tryserver from the slave so you can have the RSA fingerprint of the "tryserver" in your slave's ~/.ssh/know_hosts

• LINUX
  • buildbot create-slave slave hera:12345 lin1 lin1
• MAC
  • /usr/local/bin/buildbot create-slave /opt/slave hera:12345 mac1 mac1
• WINDOWS
  • Run c:\mozilla-build\start-msvc8.bat to start
  • /c/Python24/scripts/buildbot create-slave /c/slave/ hera:12345 win1 win1

Start

• LINUX
  • buildbot start ~/slave
• MAC
  • /usr/local/bin/buildbot start /opt/slave
• WINDOWS ("twistd.pid" does not get created)
  • Open MozBuild
  • /c/Python24/scripts/buildbot start /c/slave/ &//make note of the ampersand symbol, it puts the task in the background

Restart

• LINUX
  • buildbot restart ~/slave
• MAC
  • /usr/local/bin/buildbot restart /Users/adam.delyea/buildbot/slave
• WINDOWS - Kill it and then start it again since twistd.pid has not been generated
  • ps aux
  • Look for the ID for /c/Python24/scr <--- The column does not reach to show the complete path
  • kill 34567 //Put the correct ID
  • Check the section before to start the slave

Left to be implemented

• Install Windows Vista SDK on the windows machine - [1] (meanwhile mozoncifg_win32_test will have --disable-parent-controls)
• Install OpenSSH on WINDOWS slave so we won't have to rely on Remote Desktop Connection for changes that could be easily done with ssh
• Use incron instead cronjob to run "perl proccesschanges_test.pl" in the buildmaster machine
• Create the windows slave in /opt/slave <-- Armen thinks that I might have been refering to the mac machine
• The windows machine is being stupid and it seems not to be able to checkout anymore
• Create a little script in the try server to be able to create new users that can upload patches
• Not to be done until everything is completely fixed. Remove the whole master and checkout the latest code from cvs and this will allow us to create patches to reconstruct the master easily in any other place.
• Not to be done until everything else is completely fixed. The master.cfg is using mozbuild.py for unit tests instead of buildbotcustom/unittests/Steps.py which is the source code maintained by Mozilla and has the steps that come in mozbuild.py
• Not to be done until everything is completely fixed. Unify the version of buildbot accross all slaves

News

• There is unified form that allows you to trigger builds with unit tests and without
• Created universal patch for testing purposes since it "always" gets applied

Known Errors/Issues

• The WINDOWS machines seems to don't delete the source code and the objdir. This is something new.
• In the WINDOWS machines be sure to do not create a slave in a path that has spaces or will fail client.mk#427
• Builbot Bug 155 and Builbot Bug 85 - You have to restart the master if you see in the waterfall view machines that are IDLE but with pending builds
• The UI to upload the patches, sometimes shows the "hg" form which is not good to show if we are not using it

Fixed

• Add the ssh keys to the tryserver account, the public keys of each slave is under .ssh on the tryserver
• The SCP_STRING points to "data" on the "tryserver" account which is an alias to "data" on "buildreturn" account
• There has been clean out of files in the buildmaster account
• The builds are uploaded to the "data" folder on buildreturn account instead of the "builds" folder on the tryserver account
• The index.html only point to sendchange_test.cgi which allows you to do a build with and without unit tests
• "patches_test" is not in use anymore
• Some steps of Linux were being used for Windows and Mac
• Fixed in Windows machine - client.mk:427: *** The mozilla directory cannot be located in a path with spaces.. Stop. [2]
• Fixed in Mac machine - We have added to buildbotcustom/env.py the PATH environment value because it was missing /opt/local/bin checking for GLIB - version >= 1.2.0... no [3]
• We always do clean builds. We do not need clobbering * In "master.cfg", do we want clobbering? what are these killing for #392 to #408?