Difference between revisions of "Mozilla BuildBot Trending/Patches"
m |
|||
(6 intermediate revisions by one other user not shown) | |||
Line 1: | Line 1: | ||
+ | ==Other tests which need to be refined== | ||
+ | http://mxr.mozilla.org/mozilla-central/source/netwerk/test/TestBlockingSocket.cpp | ||
+ | |||
+ | ==Timestamp Format== | ||
+ | I think it would be a good idea to put my timestamp format in a single place. I plan on using the format <code>YYYYMMDD-HHMMSS</code>. This can be generated with <code> date +%Y%m%d-%H%M%S </code> | ||
+ | |||
+ | I am thinking to assist the parser's understanding of log files, it would be a good idea to have these prefixes for each timestamp: | ||
+ | * TOS - Timed Output Start | ||
+ | * TOF - Timed Output Finish | ||
+ | * MLTOS - Multi-Line Output Start | ||
+ | * MLTOF - Multi-Line Output Finish | ||
+ | |||
+ | ==Second attempt at timestamps== | ||
+ | This patch is my second attempt at doing timestamps. This patch lives in the build and test infrastructure instead of BuildBot. The first modification is to test_all.sh to add a parameter to specify the date format for timestamps. It will be the string passed to the terminal program 'date'. You can specify this as a parameter to test_all or by setting an environmental variable "GNUDATE_TIME_FORMAT". I am not modifying test_one.sh because I don't know if it is used in the automated test running, it seems to be there for developer usage. Here is the first stage of my patch | ||
+ | |||
+ | <pre> | ||
+ | diff -r 8feaa59e2c54 config/rules.mk | ||
+ | --- a/config/rules.mk Wed Feb 11 14:51:06 2009 +0100 | ||
+ | +++ b/config/rules.mk Wed Feb 11 11:40:27 2009 -0500 | ||
+ | @@ -210,6 +210,8 @@ | ||
+ | check:: | ||
+ | @$(EXIT_ON_ERROR) \ | ||
+ | for f in $(subst .cpp,,$(CPP_UNIT_TESTS)); do \ | ||
+ | + export GNUDATE_TIME_FORMAT ; \ | ||
+ | + echo -n "`date $GNUDATE_TIME_FORMAT` | "; \ | ||
+ | XPCOM_DEBUG_BREAK=stack-and-abort $(RUN_TEST_PROGRAM) $(DIST)/bin/$$f; \ | ||
+ | done | ||
+ | |||
+ | diff -r 8feaa59e2c54 js/src/config/autoconf.mk.in | ||
+ | --- a/js/src/config/autoconf.mk.in Wed Feb 11 14:51:06 2009 +0100 | ||
+ | +++ b/js/src/config/autoconf.mk.in Wed Feb 11 11:40:27 2009 -0500 | ||
+ | @@ -358,3 +358,4 @@ | ||
+ | ENABLE_JIT = @ENABLE_JIT@ | ||
+ | NANOJIT_ARCH = @NANOJIT_ARCH@ | ||
+ | HAVE_ARM_SIMD= @HAVE_ARM_SIMD@ | ||
+ | + | ||
+ | diff -r 8feaa59e2c54 js/src/config/rules.mk | ||
+ | --- a/js/src/config/rules.mk Wed Feb 11 14:51:06 2009 +0100 | ||
+ | +++ b/js/src/config/rules.mk Wed Feb 11 11:40:27 2009 -0500 | ||
+ | @@ -210,6 +210,8 @@ | ||
+ | check:: | ||
+ | @$(EXIT_ON_ERROR) \ | ||
+ | for f in $(subst .cpp,,$(CPP_UNIT_TESTS)); do \ | ||
+ | + export GNUDATE_TIME_FORMAT ; \ | ||
+ | + echo -n "`date $GNUDATE_TIME_FORMAT` | "; \ | ||
+ | XPCOM_DEBUG_BREAK=stack-and-abort $(RUN_TEST_PROGRAM) $(DIST)/bin/$$f; \ | ||
+ | done | ||
+ | |||
+ | diff -r 8feaa59e2c54 testing/xpcshell/test_all.sh | ||
+ | --- a/testing/xpcshell/test_all.sh Wed Feb 11 14:51:06 2009 +0100 | ||
+ | +++ b/testing/xpcshell/test_all.sh Wed Feb 11 11:40:27 2009 -0500 | ||
+ | @@ -77,6 +77,12 @@ | ||
+ | testdir=. | ||
+ | fi | ||
+ | |||
+ | +# Set this timestamp format to prepend tests. If you intend to set this | ||
+ | +# make sure you are either setting a test directory. | ||
+ | +timefmt="$5" | ||
+ | +if [ "x$timefmt" = "x" ]; then | ||
+ | + timefmt=$GNUDATE_TIME_FORMAT | ||
+ | +fi | ||
+ | |||
+ | ############################### | ||
+ | # SETUP FOR RUNNING THE TESTS # | ||
+ | @@ -90,6 +96,7 @@ | ||
+ | for h in $testdir/head_*.js | ||
+ | do | ||
+ | if [ -f $h ]; then | ||
+ | + echo -n "`date $timefmt` | "; | ||
+ | headfiles="$headfiles -f $h" | ||
+ | fi | ||
+ | done | ||
+ | @@ -101,6 +108,7 @@ | ||
+ | for t in $testdir/tail_*.js | ||
+ | do | ||
+ | if [ -f $t ]; then | ||
+ | + echo -n "`date $timefmt` | "; | ||
+ | tailfiles="$tailfiles -f $t" | ||
+ | fi | ||
+ | done | ||
+ | @@ -112,6 +120,7 @@ | ||
+ | |||
+ | for t in $testdir/test_*.js | ||
+ | do | ||
+ | + echo -n "`date $timefmt` | " | ||
+ | NATIVE_TOPSRCDIR="$native_topsrcdir" TOPSRCDIR="$topsrcdir" $xpcshell -j -s $headfiles -f $t $tailfiles 2> $t.log 1>&2 | ||
+ | rv="$?" | ||
+ | if [ ! "$rv" = "0" -o \ | ||
+ | </pre> | ||
+ | |||
==First attempt at timestamps== | ==First attempt at timestamps== | ||
This patch can be applied to buildbot-0.7.9 to enable build-logs to use the format | This patch can be applied to buildbot-0.7.9 to enable build-logs to use the format | ||
Line 37: | Line 127: | ||
This patch, applied to the previous patch changes the behaviour of the timestamps. Instead of having all information on ever line it only puts a timestamp on each line. Information that is constant throughout the run are only inserted once | This patch, applied to the previous patch changes the behaviour of the timestamps. Instead of having all information on ever line it only puts a timestamp on each line. Information that is constant throughout the run are only inserted once | ||
− | |||
− | |||
<pre>diff -r 9a6c6b742a5a buildbot/buildbot/status/builder.py | <pre>diff -r 9a6c6b742a5a buildbot/buildbot/status/builder.py | ||
--- a/buildbot/buildbot/status/builder.py Sat Feb 07 12:22:37 2009 -0500 | --- a/buildbot/buildbot/status/builder.py Sat Feb 07 12:22:37 2009 -0500 | ||
Line 78: | Line 166: | ||
def addHeader(self, text): | def addHeader(self, text): | ||
self.addEntry(HEADER, text)</pre> | self.addEntry(HEADER, text)</pre> | ||
+ | |||
+ | ==MozBuild.py and Master.cfg== | ||
+ | This patch adds my master.cfg I am using locally on my OSX machine and a modified mozbuild.py. Mozbuild.py contains all the buildbot steps used to test Mozilla. I have removed all non-OSX applicable code to minimize errors. | ||
+ | <pre> | ||
+ | diff -r 9e48e21dd4f6 buildbot/buildbot.egg-info/SOURCES.txt | ||
+ | --- a/buildbot/buildbot.egg-info/SOURCES.txt Thu Feb 05 00:39:57 2009 -0500 | ||
+ | +++ b/buildbot/buildbot.egg-info/SOURCES.txt Sat Feb 07 12:21:02 2009 -0500 | ||
+ | @@ -16,6 +16,7 @@ | ||
+ | buildbot/locks.py | ||
+ | buildbot/manhole.py | ||
+ | buildbot/master.py | ||
+ | +buildbot/mozbuild.py | ||
+ | buildbot/pbutil.py | ||
+ | buildbot/scheduler.py | ||
+ | buildbot/sourcestamp.py | ||
+ | diff -r 9e48e21dd4f6 buildbot/buildbot/mozbuild.py | ||
+ | --- /dev/null Thu Jan 01 00:00:00 1970 +0000 | ||
+ | +++ b/buildbot/buildbot/mozbuild.py Sat Feb 07 12:21:02 2009 -0500 | ||
+ | @@ -0,0 +1,345 @@ | ||
+ | +# -*- Python -*- | ||
+ | + | ||
+ | +from buildbot import steps | ||
+ | +from buildbot.steps.shell import ShellCommand | ||
+ | +from buildbot.status.builder import SUCCESS, WARNINGS, FAILURE, SKIPPED, EXCEPTION, HEADER | ||
+ | +import re | ||
+ | +import os | ||
+ | +import copy | ||
+ | + | ||
+ | +MozillaEnvironments = { } | ||
+ | + | ||
+ | +MozillaEnvironments['osx'] = { | ||
+ | + "MOZ_NO_REMOTE": '1', | ||
+ | + "NO_EM_RESTART": '1', | ||
+ | + "XPCOM_DEBUG_BREAK": 'warn', | ||
+ | + "CVS_RSH": 'ssh' | ||
+ | +} | ||
+ | + | ||
+ | + | ||
+ | + | ||
+ | +class ShellCommandReportTimeout(ShellCommand): | ||
+ | + """We subclass ShellCommand so that we can bubble up the timeout errors | ||
+ | + to tinderbox that normally only get appended to the buildbot slave logs. | ||
+ | + """ | ||
+ | + | ||
+ | + def evaluateCommand(self, cmd): | ||
+ | + superResult = ShellCommand.evaluateCommand(self, cmd) | ||
+ | + for line in cmd.logs['stdio'].readlines(channel=HEADER): | ||
+ | + if "command timed out" in line: | ||
+ | + self.addCompleteLog('timeout', | ||
+ | + 'buildbot.slave.commands.TimeoutError: ' + | ||
+ | + line + | ||
+ | + "TinderboxPrint: " + | ||
+ | + self.name + ' <em class="testfail">timeout</em><br/>\n') | ||
+ | + return WARNINGS | ||
+ | + return superResult | ||
+ | + | ||
+ | +class CreateDir(ShellCommandReportTimeout): | ||
+ | + name = "create dir" | ||
+ | + haltOnFailure = False | ||
+ | + warnOnFailure = True | ||
+ | + | ||
+ | + def __init__(self, **kwargs): | ||
+ | + if 'platform' not in kwargs: | ||
+ | + return FAILURE | ||
+ | + self.platform = kwargs['platform'] | ||
+ | + if 'dir' in kwargs: | ||
+ | + self.dir = kwargs['dir'] | ||
+ | + if self.platform.startswith('win'): | ||
+ | + self.command = r'if not exist ' + self.dir + r' mkdir ' + self.dir | ||
+ | + else: | ||
+ | + self.command = ['mkdir', '-p', self.dir] | ||
+ | + ShellCommandReportTimeout.__init__(self, **kwargs) | ||
+ | + | ||
+ | +class TinderboxShellCommand(ShellCommand): | ||
+ | + haltOnFailure = False | ||
+ | + | ||
+ | + def evaluateCommand(self, cmd): | ||
+ | + return SUCCESS | ||
+ | + | ||
+ | +class MozillaCheck(ShellCommandReportTimeout): | ||
+ | + name = "check" | ||
+ | + warnOnFailure = True | ||
+ | + description = ["checking"] | ||
+ | + descriptionDone = ["check complete"] | ||
+ | + command = ["make", "-k", "check"] | ||
+ | + | ||
+ | + def createSummary(self, log): | ||
+ | + passCount = 0 | ||
+ | + failCount = 0 | ||
+ | + for line in log: | ||
+ | + if "PASS" in line: | ||
+ | + passCount += 1 | ||
+ | + if "FAIL" in line: | ||
+ | + failCount += 1 | ||
+ | + summary = "TinderboxPrint: TUnit<br/>" + str(passCount) + "/" + str(failCount) + "\n" | ||
+ | + self.addCompleteLog('summary', summary) | ||
+ | + | ||
+ | + def evaluateCommand(self, cmd): | ||
+ | + superResult = ShellCommandReportTimeout.evaluateCommand(self, cmd) | ||
+ | + if SUCCESS != superResult: | ||
+ | + return WARNINGS | ||
+ | + if None != re.search('FAIL', cmd.logs['stdio'].getText()): | ||
+ | + return WARNINGS | ||
+ | + return SUCCESS | ||
+ | + | ||
+ | +class MozillaReftest(ShellCommandReportTimeout): | ||
+ | + warnOnFailure = True | ||
+ | + name = "reftest" | ||
+ | + description = ["reftest"] | ||
+ | + descriptionDone = ["reftest complete"] | ||
+ | + | ||
+ | + def createSummary(self, log): | ||
+ | + testCount = 0 | ||
+ | + passCount = 0 | ||
+ | + failCount = 0 | ||
+ | + knownFailCount = 0 | ||
+ | + for line in log: | ||
+ | + if "REFTEST" not in line: | ||
+ | + continue | ||
+ | + if "IMAGE" in line: | ||
+ | + continue | ||
+ | + if "RESULT EXPECTED TO BE RANDOM" in line: | ||
+ | + continue | ||
+ | + testCount += 1 | ||
+ | + if "UNEXPECTED" in line: | ||
+ | + failCount += 1 | ||
+ | + continue | ||
+ | + if "KNOWN FAIL" in line: | ||
+ | + knownFailCount += 1 | ||
+ | + else: | ||
+ | + passCount += 1 | ||
+ | + summary = "TinderboxPrint: " + self.name + "<br/>" + str(passCount) + \ | ||
+ | + "/" + str(failCount) + "/" + str(knownFailCount) + "\n" | ||
+ | + self.addCompleteLog('summary', summary) | ||
+ | + | ||
+ | + def evaluateCommand(self, cmd): | ||
+ | + superResult = ShellCommandReportTimeout.evaluateCommand(self, cmd) | ||
+ | + if SUCCESS != superResult: | ||
+ | + return WARNINGS | ||
+ | + if re.search('UNEXPECTED', cmd.logs['stdio'].getText()): | ||
+ | + return WARNINGS | ||
+ | + return SUCCESS | ||
+ | + | ||
+ | +class MozillaOSXReftest(MozillaReftest): | ||
+ | + command = ["../../objdir/dist/Minefield.app/Contents/MacOS/firefox", | ||
+ | + "-console", | ||
+ | + "-P", | ||
+ | + "default", | ||
+ | + "-reftest", | ||
+ | + "reftest.list"] | ||
+ | + | ||
+ | +class MozillaCrashtest(MozillaReftest): | ||
+ | + name = "crashtest" | ||
+ | + description = ["crashtest"] | ||
+ | + descriptionDone = ["crashtest complete"] | ||
+ | + | ||
+ | +class MozillaOSXCrashtest(MozillaCrashtest): | ||
+ | + command = ["../../objdir/dist/Minefield.app/Contents/MacOS/firefox", | ||
+ | + "-console", | ||
+ | + "-P", | ||
+ | + "default", | ||
+ | + "-reftest", | ||
+ | + "crashtests.list"] | ||
+ | + | ||
+ | +class MozillaMochitest(ShellCommandReportTimeout): | ||
+ | + name = "mochitest" | ||
+ | + warnOnFailure = True | ||
+ | + description = ["mochitest"] | ||
+ | + descriptionDone = ["mochitest complete"] | ||
+ | + command = ["python", | ||
+ | + "runtests.py", | ||
+ | + "--appname=../../../dist/bin/firefox", | ||
+ | + "--autorun", | ||
+ | + "--console-level=INFO", | ||
+ | + "--close-when-done"] | ||
+ | + | ||
+ | + def createSummary(self, log): | ||
+ | + passCount = 0 | ||
+ | + failCount = 0 | ||
+ | + todoCount = 0 | ||
+ | + for line in log: | ||
+ | + if "INFO Passed:" in line: | ||
+ | + passCount = int(line.split()[-1]) | ||
+ | + if "INFO Failed:" in line: | ||
+ | + failCount = int(line.split()[-1]) | ||
+ | + if "INFO Todo:" in line: | ||
+ | + todoCount = int(line.split()[-1]) | ||
+ | + summary = "TinderboxPrint: mochitest<br/>" | ||
+ | + if not (passCount + failCount + todoCount): | ||
+ | + summary += "FAIL\n" | ||
+ | + else: | ||
+ | + summary += str(passCount) + "/" + str(failCount) + "/" + str(todoCount) + "\n" | ||
+ | + self.addCompleteLog('summary', summary) | ||
+ | + | ||
+ | + def evaluateCommand(self, cmd): | ||
+ | + superResult = ShellCommandReportTimeout.evaluateCommand(self, cmd) | ||
+ | + if SUCCESS != superResult: | ||
+ | + return WARNINGS | ||
+ | + if re.search('ERROR FAIL', cmd.logs['stdio'].getText()): | ||
+ | + return WARNINGS | ||
+ | + if re.search('ERROR TODO WORKED', cmd.logs['stdio'].getText()): | ||
+ | + return WARNINGS | ||
+ | + if re.search('FAIL Exited', cmd.logs['stdio'].getText()): | ||
+ | + return WARNINGS | ||
+ | + if not re.search('INFO PASS', cmd.logs['stdio'].getText()): | ||
+ | + return WARNINGS | ||
+ | + return SUCCESS | ||
+ | + | ||
+ | +class MozillaOSXMochitest(MozillaMochitest): | ||
+ | + command = ["python", | ||
+ | + "runtests.py", | ||
+ | + "--appname=../../../objdir/dist/Minefield.app/Contents/MacOS/firefox", | ||
+ | + "--autorun", | ||
+ | + "--console-level=INFO", | ||
+ | + "--close-when-done"] | ||
+ | + | ||
+ | +class MozillaMochichrome(ShellCommandReportTimeout): | ||
+ | + name = "mochichrome" | ||
+ | + warnOnFailure = True | ||
+ | + description = ["mochichrome"] | ||
+ | + descriptionDone = ["mochichrome complete"] | ||
+ | + command = ["python", | ||
+ | + "runtests.py", | ||
+ | + "--appname=objdir/dist/bin/firefox", | ||
+ | + "--chrome", | ||
+ | + "--autorun", | ||
+ | + "--console-level=INFO", | ||
+ | + "--close-when-done"] | ||
+ | + | ||
+ | + def createSummary(self, log): | ||
+ | + passCount = 0 | ||
+ | + failCount = 0 | ||
+ | + todoCount = 0 | ||
+ | + for line in log: | ||
+ | + if "INFO Passed:" in line: | ||
+ | + passCount = int(line.split()[-1]) | ||
+ | + if "INFO Failed:" in line: | ||
+ | + failCount = int(line.split()[-1]) | ||
+ | + if "INFO Todo:" in line: | ||
+ | + todoCount = int(line.split()[-1]) | ||
+ | + summary = "TinderboxPrint: chrome<br/>" | ||
+ | + if not (passCount + failCount + todoCount): | ||
+ | + summary += "FAIL\n" | ||
+ | + else: | ||
+ | + summary += str(passCount) + "/" + str(failCount) + "/" + str(todoCount) + "\n" | ||
+ | + self.addCompleteLog('summary', summary) | ||
+ | + | ||
+ | + def evaluateCommand(self, cmd): | ||
+ | + superResult = ShellCommandReportTimeout.evaluateCommand(self, cmd) | ||
+ | + if SUCCESS != superResult: | ||
+ | + return WARNINGS | ||
+ | + if re.search('ERROR FAIL', cmd.logs['stdio'].getText()): | ||
+ | + return WARNINGS | ||
+ | + if re.search('FAIL Exited', cmd.logs['stdio'].getText()): | ||
+ | + return WARNINGS | ||
+ | + if not re.search('INFO PASS', cmd.logs['stdio'].getText()): | ||
+ | + return WARNINGS | ||
+ | + return SUCCESS | ||
+ | + | ||
+ | + | ||
+ | +class MozillaOSXMochichrome(MozillaMochichrome): | ||
+ | + command = ["python", | ||
+ | + "runtests.py", | ||
+ | + "--appname=../../../objdir/dist/Minefield.app/Contents/MacOS/firefox", | ||
+ | + "--chrome", | ||
+ | + "--autorun", | ||
+ | + "--console-level=INFO", | ||
+ | + "--close-when-done"] | ||
+ | + | ||
+ | +class MozillaBrowserChromeTest(ShellCommandReportTimeout): | ||
+ | + name = "browser chrome test" | ||
+ | + warnOnFailure = True | ||
+ | + description = ["browser chrome test"] | ||
+ | + descriptionDone = ["browser chrome test complete"] | ||
+ | + command = ["python", | ||
+ | + "runtests.py", | ||
+ | + "--appname=../../../dist/bin/firefox", | ||
+ | + "--autorun", | ||
+ | + "--browser-chrome", | ||
+ | + "--close-when-done"] | ||
+ | + | ||
+ | + def createSummary(self, log): | ||
+ | + passCount = 0 | ||
+ | + failCount = 0 | ||
+ | + todoCount = 0 | ||
+ | + for line in log: | ||
+ | + if "Pass:" in line: | ||
+ | + passCount = int(line.split()[-1]) | ||
+ | + if "Fail:" in line: | ||
+ | + failCount = int(line.split()[-1]) | ||
+ | + if "Todo:" in line: | ||
+ | + todoCount = int(line.split()[-1]) | ||
+ | + summary = "TinderboxPrint: browser<br/>" | ||
+ | + if not (passCount + failCount + todoCount): | ||
+ | + summary += "FAIL\n" | ||
+ | + else: | ||
+ | + summary += str(passCount) + "/" + str(failCount) + "/" + str(todoCount) + "\n" | ||
+ | + self.addCompleteLog('summary', summary) | ||
+ | + | ||
+ | + def evaluateCommand(self, cmd): | ||
+ | + superResult = ShellCommandReportTimeout.evaluateCommand(self, cmd) | ||
+ | + if SUCCESS != superResult: | ||
+ | + return WARNINGS | ||
+ | + if re.search('FAIL -', cmd.logs['stdio'].getText()): | ||
+ | + return WARNINGS | ||
+ | + if re.search('FAIL Exited', cmd.logs['stdio'].getText()): | ||
+ | + return WARNINGS | ||
+ | + return SUCCESS | ||
+ | + | ||
+ | +class MozillaOSXBrowserChromeTest(MozillaBrowserChromeTest): | ||
+ | + command = ["python", | ||
+ | + "runtests.py", | ||
+ | + "--appname=../../../objdir/dist/Minefield.app/Contents/MacOS/firefox", | ||
+ | + "--autorun", | ||
+ | + "--browser-chrome", | ||
+ | + "--close-when-done"] | ||
+ | + | ||
+ | +class MozillaA11YTest(MozillaMochichrome): | ||
+ | + name = "a11y test" | ||
+ | + warnOnFailure = True | ||
+ | + description = ["a11y test"] | ||
+ | + descriptionDone = ["a11y test complete"] | ||
+ | + command = ["python", | ||
+ | + "runtests.py", | ||
+ | + "--appname=objdir/dist/bin/firefox", | ||
+ | + "--console-level=INFO", | ||
+ | + "--autorun", | ||
+ | + "--a11y", | ||
+ | + "--close-when-done"] | ||
+ | + | ||
+ | + def createSummary(self, log): | ||
+ | + passCount = 0 | ||
+ | + failCount = 0 | ||
+ | + todoCount = 0 | ||
+ | + for line in log: | ||
+ | + if "INFO Passed:" in line: | ||
+ | + passCount = int(line.split()[-1]) | ||
+ | + if "INFO Failed:" in line: | ||
+ | + failCount = int(line.split()[-1]) | ||
+ | + if "INFO Todo:" in line: | ||
+ | + todoCount = int(line.split()[-1]) | ||
+ | + summary = "TinderboxPrint: a11y<br/>" | ||
+ | + if not (passCount + failCount + todoCount): | ||
+ | + summary += "FAIL\n" | ||
+ | + else: | ||
+ | + summary += str(passCount) + "/" + str(failCount) + "/" + str(todoCount) + "\n" | ||
+ | + self.addCompleteLog('summary', summary) | ||
+ | + | ||
+ | + | ||
+ | +class MozillaOSXA11YTest(MozillaA11YTest): | ||
+ | + command = ["python", | ||
+ | + "runtests.py", | ||
+ | + "--appname=../../../dist/Minefield.app/Contents/MacOS/firefox", | ||
+ | + "--a11y", | ||
+ | + "--autorun", | ||
+ | + "--console-level=INFO", | ||
+ | + "--close-when-done"] | ||
+ | + | ||
+ | +class CreateProfile(ShellCommandReportTimeout): | ||
+ | + name = "create profile" | ||
+ | + warnOnFailure = True | ||
+ | + description = ["create profile"] | ||
+ | + descriptionDone = ["create profile complete"] | ||
+ | + command = r'python testing/tools/profiles/createTestingProfile.py --clobber --binary objdir/dist/bin/firefox' | ||
+ | diff -r 9e48e21dd4f6 master.cfg | ||
+ | --- /dev/null Thu Jan 01 00:00:00 1970 +0000 | ||
+ | +++ b/master.cfg Sat Feb 07 12:21:02 2009 -0500 | ||
+ | @@ -0,0 +1,238 @@ | ||
+ | +# -*- python -*- | ||
+ | +# ex: set syntax=python: | ||
+ | + | ||
+ | +# This is a sample buildmaster config file. It must be installed as | ||
+ | +# 'master.cfg' in your buildmaster's base directory (although the filename | ||
+ | +# can be changed with the --basedir option to 'mktap buildbot master'). | ||
+ | + | ||
+ | +# It has one job: define a dictionary named BuildmasterConfig. This | ||
+ | +# dictionary has a variety of keys to control different aspects of the | ||
+ | +# buildmaster. They are documented in docs/config.xhtml . | ||
+ | + | ||
+ | + | ||
+ | +# This is the dictionary that the buildmaster pays attention to. We also use | ||
+ | +# a shorter alias to save typing. | ||
+ | +c = BuildmasterConfig = {} | ||
+ | + | ||
+ | +####### BUILDSLAVES | ||
+ | + | ||
+ | +# the 'slaves' list defines the set of allowable buildslaves. Each element is | ||
+ | +# a tuple of bot-name and bot-password. These correspond to values given to | ||
+ | +# the buildslave's mktap invocation. | ||
+ | +from buildbot.buildslave import BuildSlave | ||
+ | +#c['slaves'] = [BuildSlave("p6t-slave", "mozilla"), BuildSlave("macbook-slave", "mozilla"), BuildSlave("neb-slave", "mozilla")] | ||
+ | +c['slaves'] = [BuildSlave("local-slave", "mozilla")] | ||
+ | + | ||
+ | +# to limit to two concurrent builds on a slave, use | ||
+ | +# c['slaves'] = [BuildSlave("bot1name", "bot1passwd", max_builds=2)] | ||
+ | + | ||
+ | + | ||
+ | +# 'slavePortnum' defines the TCP port to listen on. This must match the value | ||
+ | +# configured into the buildslaves (with their --master option) | ||
+ | + | ||
+ | +c['slavePortnum'] = 9989 | ||
+ | + | ||
+ | +####### CHANGESOURCES | ||
+ | + | ||
+ | +# the 'change_source' setting tells the buildmaster how it should find out | ||
+ | +# about source code changes. Any class which implements IChangeSource can be | ||
+ | +# put here: there are several in buildbot/changes/*.py to choose from. | ||
+ | + | ||
+ | +#from buildbot.changes.pb import PBChangeSource | ||
+ | +#c['change_source'] = [] | ||
+ | +#c['change_source'].append(PBChangeSource()) | ||
+ | + | ||
+ | +# For example, if you had CVSToys installed on your repository, and your | ||
+ | +# CVSROOT/freshcfg file had an entry like this: | ||
+ | +#pb = ConfigurationSet([ | ||
+ | +# (None, None, None, PBService(userpass=('foo', 'bar'), port=4519)), | ||
+ | +# ]) | ||
+ | + | ||
+ | +# then you could use the following buildmaster Change Source to subscribe to | ||
+ | +# the FreshCVS daemon and be notified on every commit: | ||
+ | +# | ||
+ | +#from buildbot.changes.freshcvs import FreshCVSSource | ||
+ | +#fc_source = FreshCVSSource("cvs.example.com", 4519, "foo", "bar") | ||
+ | +#c['change_source'] = fc_source | ||
+ | + | ||
+ | +# or, use a PBChangeSource, and then have your repository's commit script run | ||
+ | +# 'buildbot sendchange', or use contrib/svn_buildbot.py, or | ||
+ | +# contrib/arch_buildbot.py : | ||
+ | +# | ||
+ | +#from buildbot.changes.pb import PBChangeSource | ||
+ | +#c['change_source'] = PBChangeSource() | ||
+ | + | ||
+ | + | ||
+ | + | ||
+ | +####### SCHEDULERS | ||
+ | + | ||
+ | +## configure the Schedulers | ||
+ | + | ||
+ | +from buildbot.scheduler import Scheduler | ||
+ | +c['schedulers'] = [] | ||
+ | +#c['schedulers'].append(Scheduler(name="all", branch=None, | ||
+ | +# treeStableTimer=5, | ||
+ | +# builderNames=["buildbot-f10", "buildbot-osx", "buildbot-xp"])) | ||
+ | +# | ||
+ | + | ||
+ | +####### BUILDERS | ||
+ | + | ||
+ | +# the 'builders' list defines the Builders. Each one is configured with a | ||
+ | +# dictionary, using the following keys: | ||
+ | +# name (required): the name used to describe this bilder | ||
+ | +# slavename (required): which slave to use, must appear in c['bots'] | ||
+ | +# builddir (required): which subdirectory to run the builder in | ||
+ | +# factory (required): a BuildFactory to define how the build is run | ||
+ | +# periodicBuildTime (optional): if set, force a build every N seconds | ||
+ | + | ||
+ | +# buildbot/process/factory.py provides several BuildFactory classes you can | ||
+ | +# start with, which implement build processes for common targets (GNU | ||
+ | +# autoconf projects, CPAN perl modules, etc). The factory.BuildFactory is the | ||
+ | +# base class, and is configured with a series of BuildSteps. When the build | ||
+ | +# is run, the appropriate buildslave is told to execute each Step in turn. | ||
+ | + | ||
+ | +# the first BuildStep is typically responsible for obtaining a copy of the | ||
+ | +# sources. There are source-obtaining Steps in buildbot/steps/source.py for | ||
+ | +# CVS, SVN, and others. | ||
+ | + | ||
+ | +#John's Simple Factory | ||
+ | +from buildbot.process import factory | ||
+ | +from buildbot.steps.shell import ShellCommand, Compile | ||
+ | +from buildbot.steps.source import Mercurial | ||
+ | +f1 = factory.BuildFactory() | ||
+ | +import os | ||
+ | +f1.addStep(ShellCommand, name='current date-time', command='~/pytime.py') | ||
+ | +f1.addStep(Mercurial, repourl="http://hg.mozilla.org/mozilla-central/", mode='update'); | ||
+ | +f1.addStep(ShellCommand, flunkOnFail=False, haltOnFail=False, name='clean-obj', command='rm -rf obj',) | ||
+ | +f1.addStep(Compile, name='build', command=['make', '-f', 'client.mk', 'build']) | ||
+ | + | ||
+ | + | ||
+ | +## | ||
+ | +## Mac OS X | ||
+ | +## | ||
+ | +from buildbot.mozbuild import * | ||
+ | + | ||
+ | +osxFactory = factory.BuildFactory() | ||
+ | + | ||
+ | +osxFactory.addStep(ShellCommand, name='current date-time', command='~/pytime.py') | ||
+ | +osxFactory.addStep(ShellCommand, name="mozconfig contents", | ||
+ | + command=["cat","/home/jhford/.mozconfig"]) | ||
+ | +osxFactory.addStep(Mercurial, repourl="http://hg.mozilla.org/mozilla-central/", mode='update'); | ||
+ | +osxFactory.addStep(ShellCommand, flunkOnFail=False, haltOnFail=False, name='clean-obj', command='rm -rf obj',) | ||
+ | +osxFactory.addStep(Compile, name='build', command=['make', '-f', 'client.mk', 'build']) | ||
+ | +osxFactory.addStep(ShellCommand, name='check', workdir='build/objdir', command='make -k check') | ||
+ | +osxFactory.addStep(CreateProfile, | ||
+ | + warnOnWarnings=True, | ||
+ | + env=MozillaEnvironments['osx'], | ||
+ | + clobber=True) | ||
+ | +osxFactory.addStep(MozillaOSXReftest, warnOnWarnings=True, | ||
+ | + workdir="build/layout/reftests", | ||
+ | + env=MozillaEnvironments['osx']) | ||
+ | +osxFactory.addStep(MozillaOSXCrashtest, warnOnWarnings=True, | ||
+ | + workdir="build/testing/crashtest", | ||
+ | + env=MozillaEnvironments['osx']) | ||
+ | +#osxFactory.addStep(MozillaMochitest, warnOnWarnings=True, | ||
+ | +# workdir="build/objdir/_tests/testing/mochitest", | ||
+ | +# env=MozillaEnvironments['osx']) | ||
+ | +#osxFactory.addStep(MozillaMochichrome, warnOnWarnings=True, | ||
+ | +# workdir="build/objdir/_tests/testing/mochitest", | ||
+ | +# env=MozillaEnvironments['osx']) | ||
+ | +#osxFactory.addStep(MozillaBrowserChromeTest, warnOnWarnings=True, | ||
+ | +# workdir="build/objdir/_tests/testing/mochitest", | ||
+ | +# env=MozillaEnvironments['osx']) | ||
+ | + | ||
+ | + | ||
+ | +b1 = {'name': "local-osx", | ||
+ | + 'slavename': "local-slave", | ||
+ | + 'builddir': "osx", | ||
+ | + 'factory': f1, | ||
+ | + } | ||
+ | + | ||
+ | +mozbuilder = {'name': "mozilla-osx", | ||
+ | + 'slavename': "local-slave", | ||
+ | + 'builddir': "moz-osx", | ||
+ | + 'factory': osxFactory, | ||
+ | + } | ||
+ | + | ||
+ | + | ||
+ | +b2 = {'name': "buildbot-f10", | ||
+ | + 'slavename': "p6t-slave", | ||
+ | + 'builddir': "linux", | ||
+ | + 'factory': f1, | ||
+ | + } | ||
+ | + | ||
+ | +#b3 = {'name': "buildbot-xp", | ||
+ | +# 'slavename': "neb-slave", | ||
+ | +# 'builddir': "xp", | ||
+ | +# 'factory': f1, | ||
+ | +# } | ||
+ | + | ||
+ | + | ||
+ | +c['builders'] = [] | ||
+ | +c['builders'].append(b1) | ||
+ | +c['builders'].append(mozbuilder) | ||
+ | +#c['builders'].append(b2) | ||
+ | +#c['builders'].append(b3) | ||
+ | + | ||
+ | + | ||
+ | +####### STATUS TARGETS | ||
+ | + | ||
+ | +# 'status' is a list of Status Targets. The results of each build will be | ||
+ | +# pushed to these targets. buildbot/status/*.py has a variety to choose from, | ||
+ | +# including web pages, email senders, and IRC bots. | ||
+ | + | ||
+ | +c['status'] = [] | ||
+ | + | ||
+ | +from buildbot.status import html | ||
+ | +c['status'].append(html.WebStatus(http_port=8010, allowForce=True)) | ||
+ | + | ||
+ | +from buildbot.status import mail | ||
+ | +c['status'].append(mail.MailNotifier(fromaddr="buildbot@localhost", | ||
+ | + extraRecipients=["buildlogs@johnford.info"], | ||
+ | + sendToInterestedUsers=False)) | ||
+ | + | ||
+ | +from buildbot.status import words | ||
+ | +c['status'].append(words.IRC(host="irc.mozilla.org", nick="buildbot-seneca", | ||
+ | + channels=["#seneca-build"])) | ||
+ | + | ||
+ | +from buildbot.status import client | ||
+ | +c['status'].append(client.PBListener(9988)) | ||
+ | + | ||
+ | + | ||
+ | +####### DEBUGGING OPTIONS | ||
+ | + | ||
+ | +# if you set 'debugPassword', then you can connect to the buildmaster with | ||
+ | +# the diagnostic tool in contrib/debugclient.py . From this tool, you can | ||
+ | +# manually force builds and inject changes, which may be useful for testing | ||
+ | +# your buildmaster without actually commiting changes to your repository (or | ||
+ | +# before you have a functioning 'sources' set up). The debug tool uses the | ||
+ | +# same port number as the slaves do: 'slavePortnum'. | ||
+ | + | ||
+ | +#c['debugPassword'] = "debugpassword" | ||
+ | + | ||
+ | +# if you set 'manhole', you can ssh into the buildmaster and get an | ||
+ | +# interactive python shell, which may be useful for debugging buildbot | ||
+ | +# internals. It is probably only useful for buildbot developers. You can also | ||
+ | +# use an authorized_keys file, or plain telnet. | ||
+ | +#from buildbot import manhole | ||
+ | +#c['manhole'] = manhole.PasswordManhole("tcp:9999:interface=127.0.0.1", | ||
+ | +# "admin", "password") | ||
+ | + | ||
+ | + | ||
+ | +####### PROJECT IDENTITY | ||
+ | + | ||
+ | +# the 'projectName' string will be used to describe the project that this | ||
+ | +# buildbot is working on. For example, it is used as the title of the | ||
+ | +# waterfall HTML page. The 'projectURL' string will be used to provide a link | ||
+ | +# from buildbot HTML pages to your project's home page. | ||
+ | + | ||
+ | +c['projectName'] = "Mozilla @ Seneca" | ||
+ | +c['projectURL'] = "http://zenit.senecac.on.ca/wiki" | ||
+ | + | ||
+ | +# the 'buildbotURL' string should point to the location where the buildbot's | ||
+ | +# internal web server (usually the html.Waterfall page) is visible. This | ||
+ | +# typically uses the port number set in the Waterfall 'status' entry, but | ||
+ | +# with an externally-visible host name which the buildbot cannot figure out | ||
+ | +# without some help. | ||
+ | + | ||
+ | +c['buildbotURL'] = "http://localhost:8010/" | ||
+ | </pre> |
Latest revision as of 17:15, 8 August 2010
Contents
Other tests which need to be refined
http://mxr.mozilla.org/mozilla-central/source/netwerk/test/TestBlockingSocket.cpp
Timestamp Format
I think it would be a good idea to put my timestamp format in a single place. I plan on using the format YYYYMMDD-HHMMSS
. This can be generated with date +%Y%m%d-%H%M%S
I am thinking to assist the parser's understanding of log files, it would be a good idea to have these prefixes for each timestamp:
- TOS - Timed Output Start
- TOF - Timed Output Finish
- MLTOS - Multi-Line Output Start
- MLTOF - Multi-Line Output Finish
Second attempt at timestamps
This patch is my second attempt at doing timestamps. This patch lives in the build and test infrastructure instead of BuildBot. The first modification is to test_all.sh to add a parameter to specify the date format for timestamps. It will be the string passed to the terminal program 'date'. You can specify this as a parameter to test_all or by setting an environmental variable "GNUDATE_TIME_FORMAT". I am not modifying test_one.sh because I don't know if it is used in the automated test running, it seems to be there for developer usage. Here is the first stage of my patch
diff -r 8feaa59e2c54 config/rules.mk --- a/config/rules.mk Wed Feb 11 14:51:06 2009 +0100 +++ b/config/rules.mk Wed Feb 11 11:40:27 2009 -0500 @@ -210,6 +210,8 @@ check:: @$(EXIT_ON_ERROR) \ for f in $(subst .cpp,,$(CPP_UNIT_TESTS)); do \ + export GNUDATE_TIME_FORMAT ; \ + echo -n "`date $GNUDATE_TIME_FORMAT` | "; \ XPCOM_DEBUG_BREAK=stack-and-abort $(RUN_TEST_PROGRAM) $(DIST)/bin/$$f; \ done diff -r 8feaa59e2c54 js/src/config/autoconf.mk.in --- a/js/src/config/autoconf.mk.in Wed Feb 11 14:51:06 2009 +0100 +++ b/js/src/config/autoconf.mk.in Wed Feb 11 11:40:27 2009 -0500 @@ -358,3 +358,4 @@ ENABLE_JIT = @ENABLE_JIT@ NANOJIT_ARCH = @NANOJIT_ARCH@ HAVE_ARM_SIMD= @HAVE_ARM_SIMD@ + diff -r 8feaa59e2c54 js/src/config/rules.mk --- a/js/src/config/rules.mk Wed Feb 11 14:51:06 2009 +0100 +++ b/js/src/config/rules.mk Wed Feb 11 11:40:27 2009 -0500 @@ -210,6 +210,8 @@ check:: @$(EXIT_ON_ERROR) \ for f in $(subst .cpp,,$(CPP_UNIT_TESTS)); do \ + export GNUDATE_TIME_FORMAT ; \ + echo -n "`date $GNUDATE_TIME_FORMAT` | "; \ XPCOM_DEBUG_BREAK=stack-and-abort $(RUN_TEST_PROGRAM) $(DIST)/bin/$$f; \ done diff -r 8feaa59e2c54 testing/xpcshell/test_all.sh --- a/testing/xpcshell/test_all.sh Wed Feb 11 14:51:06 2009 +0100 +++ b/testing/xpcshell/test_all.sh Wed Feb 11 11:40:27 2009 -0500 @@ -77,6 +77,12 @@ testdir=. fi +# Set this timestamp format to prepend tests. If you intend to set this +# make sure you are either setting a test directory. +timefmt="$5" +if [ "x$timefmt" = "x" ]; then + timefmt=$GNUDATE_TIME_FORMAT +fi ############################### # SETUP FOR RUNNING THE TESTS # @@ -90,6 +96,7 @@ for h in $testdir/head_*.js do if [ -f $h ]; then + echo -n "`date $timefmt` | "; headfiles="$headfiles -f $h" fi done @@ -101,6 +108,7 @@ for t in $testdir/tail_*.js do if [ -f $t ]; then + echo -n "`date $timefmt` | "; tailfiles="$tailfiles -f $t" fi done @@ -112,6 +120,7 @@ for t in $testdir/test_*.js do + echo -n "`date $timefmt` | " NATIVE_TOPSRCDIR="$native_topsrcdir" TOPSRCDIR="$topsrcdir" $xpcshell -j -s $headfiles -f $t $tailfiles 2> $t.log 1>&2 rv="$?" if [ ! "$rv" = "0" -o \
First attempt at timestamps
This patch can be applied to buildbot-0.7.9 to enable build-logs to use the format
1233811875.619101 | 'local-slave' | local-osx | 41 | pulling from http://hg.mozilla.org/mozilla-central/ 1233811875.9946001 | 'local-slave' | local-osx | 41 | searching for changes 1233811878.362067 | 'local-slave' | local-osx | 41 | adding changesets 1233811878.373631 | 'local-slave' | local-osx | 41 | adding manifests 1233811878.7630169 | 'local-slave' | local-osx | 41 | adding file changes 1233811881.238174 | 'local-slave' | local-osx | 41 | added 2 changesets with 4 changes to 4 files
The format is unixtime | build slave name | builder name | build number | output
The patch is applied against http://downloads.sourceforge.net/buildbot/buildbot-0.7.9.tar.gz
File:Timestamp.patch.zip
diff -r a9c16958d5dc buildbot/buildbot/status/builder.py --- a/buildbot/buildbot/status/builder.py Thu Feb 05 00:37:33 2009 -0500 +++ b/buildbot/buildbot/status/builder.py Thu Feb 05 00:39:45 2009 -0500 @@ -393,9 +393,9 @@ self.length += len(text) def addStdout(self, text): - self.addEntry(STDOUT, text) + self.addEntry(STDOUT, repr(time()) + ' | ' + repr(self.getStep().build.slavename) + ' | ' + self.getStep().build.getBuilder().getName() + ' | ' + repr(self.getStep().build.getNumber()) + ' | ' + text + '\n') def addStderr(self, text): - self.addEntry(STDERR, text) + self.addEntry(STDERR, repr(time()) + ' | ' + repr(self.getStep().build.slavename) + ' | ' + self.getStep().build.getBuilder().getName() + ' | ' + repr(self.getStep().build.getNumber()) + ' | ' + text + '\n') def addHeader(self, text): self.addEntry(HEADER, text)
New Timestamps Patch
This patch, applied to the previous patch changes the behaviour of the timestamps. Instead of having all information on ever line it only puts a timestamp on each line. Information that is constant throughout the run are only inserted once
diff -r 9a6c6b742a5a buildbot/buildbot/status/builder.py --- a/buildbot/buildbot/status/builder.py Sat Feb 07 12:22:37 2009 -0500 +++ b/buildbot/buildbot/status/builder.py Sat Feb 07 12:27:49 2009 -0500 @@ -6,6 +6,8 @@ from twisted.internet import reactor, defer from twisted.protocols import basic from buildbot.process.properties import Properties +# JOHNFORD +from time import time import os, shutil, sys, re, urllib, itertools from cPickle import load, dump @@ -232,6 +234,12 @@ self.runEntries = [] self.watchers = [] self.finishedWatchers = [] + # Add some information about the build to the log + self.addEntry(STDOUT, repr(time()) + ' | ' + + repr(self.getStep().build.slavename) + ' | ' + + self.getStep().build.getBuilder().getName() + ' | ' + + repr(self.getStep().build.getNumber()))# + ' | ' + #+ repr(getFilename())) def getFilename(self): return os.path.join(self.step.build.builder.basedir, self.filename) @@ -391,11 +399,11 @@ for w in self.watchers: w.logChunk(self.step.build, self.step, self, channel, text) self.length += len(text) - + def addStdout(self, text): - self.addEntry(STDOUT, repr(time()) + ' | ' + repr(self.getStep().build.slavename) + ' | ' + self.getStep().build.getBuilder().getName() + ' | ' + repr(self.getStep().build.getNumber()) + ' | ' + text + '\n') + self.addEntry(STDOUT, repr(time()) + ' | ' + text) def addStderr(self, text): - self.addEntry(STDERR, repr(time()) + ' | ' + repr(self.getStep().build.slavename) + ' | ' + self.getStep().build.getBuilder().getName() + ' | ' + repr(self.getStep().build.getNumber()) + ' | ' + text + '\n') + self.addEntry(STDERR, repr(time()) + ' | ' + text) def addHeader(self, text): self.addEntry(HEADER, text)
MozBuild.py and Master.cfg
This patch adds my master.cfg I am using locally on my OSX machine and a modified mozbuild.py. Mozbuild.py contains all the buildbot steps used to test Mozilla. I have removed all non-OSX applicable code to minimize errors.
diff -r 9e48e21dd4f6 buildbot/buildbot.egg-info/SOURCES.txt --- a/buildbot/buildbot.egg-info/SOURCES.txt Thu Feb 05 00:39:57 2009 -0500 +++ b/buildbot/buildbot.egg-info/SOURCES.txt Sat Feb 07 12:21:02 2009 -0500 @@ -16,6 +16,7 @@ buildbot/locks.py buildbot/manhole.py buildbot/master.py +buildbot/mozbuild.py buildbot/pbutil.py buildbot/scheduler.py buildbot/sourcestamp.py diff -r 9e48e21dd4f6 buildbot/buildbot/mozbuild.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/buildbot/buildbot/mozbuild.py Sat Feb 07 12:21:02 2009 -0500 @@ -0,0 +1,345 @@ +# -*- Python -*- + +from buildbot import steps +from buildbot.steps.shell import ShellCommand +from buildbot.status.builder import SUCCESS, WARNINGS, FAILURE, SKIPPED, EXCEPTION, HEADER +import re +import os +import copy + +MozillaEnvironments = { } + +MozillaEnvironments['osx'] = { + "MOZ_NO_REMOTE": '1', + "NO_EM_RESTART": '1', + "XPCOM_DEBUG_BREAK": 'warn', + "CVS_RSH": 'ssh' +} + + + +class ShellCommandReportTimeout(ShellCommand): + """We subclass ShellCommand so that we can bubble up the timeout errors + to tinderbox that normally only get appended to the buildbot slave logs. + """ + + def evaluateCommand(self, cmd): + superResult = ShellCommand.evaluateCommand(self, cmd) + for line in cmd.logs['stdio'].readlines(channel=HEADER): + if "command timed out" in line: + self.addCompleteLog('timeout', + 'buildbot.slave.commands.TimeoutError: ' + + line + + "TinderboxPrint: " + + self.name + ' <em class="testfail">timeout</em><br/>\n') + return WARNINGS + return superResult + +class CreateDir(ShellCommandReportTimeout): + name = "create dir" + haltOnFailure = False + warnOnFailure = True + + def __init__(self, **kwargs): + if 'platform' not in kwargs: + return FAILURE + self.platform = kwargs['platform'] + if 'dir' in kwargs: + self.dir = kwargs['dir'] + if self.platform.startswith('win'): + self.command = r'if not exist ' + self.dir + r' mkdir ' + self.dir + else: + self.command = ['mkdir', '-p', self.dir] + ShellCommandReportTimeout.__init__(self, **kwargs) + +class TinderboxShellCommand(ShellCommand): + haltOnFailure = False + + def evaluateCommand(self, cmd): + return SUCCESS + +class MozillaCheck(ShellCommandReportTimeout): + name = "check" + warnOnFailure = True + description = ["checking"] + descriptionDone = ["check complete"] + command = ["make", "-k", "check"] + + def createSummary(self, log): + passCount = 0 + failCount = 0 + for line in log: + if "PASS" in line: + passCount += 1 + if "FAIL" in line: + failCount += 1 + summary = "TinderboxPrint: TUnit<br/>" + str(passCount) + "/" + str(failCount) + "\n" + self.addCompleteLog('summary', summary) + + def evaluateCommand(self, cmd): + superResult = ShellCommandReportTimeout.evaluateCommand(self, cmd) + if SUCCESS != superResult: + return WARNINGS + if None != re.search('FAIL', cmd.logs['stdio'].getText()): + return WARNINGS + return SUCCESS + +class MozillaReftest(ShellCommandReportTimeout): + warnOnFailure = True + name = "reftest" + description = ["reftest"] + descriptionDone = ["reftest complete"] + + def createSummary(self, log): + testCount = 0 + passCount = 0 + failCount = 0 + knownFailCount = 0 + for line in log: + if "REFTEST" not in line: + continue + if "IMAGE" in line: + continue + if "RESULT EXPECTED TO BE RANDOM" in line: + continue + testCount += 1 + if "UNEXPECTED" in line: + failCount += 1 + continue + if "KNOWN FAIL" in line: + knownFailCount += 1 + else: + passCount += 1 + summary = "TinderboxPrint: " + self.name + "<br/>" + str(passCount) + \ + "/" + str(failCount) + "/" + str(knownFailCount) + "\n" + self.addCompleteLog('summary', summary) + + def evaluateCommand(self, cmd): + superResult = ShellCommandReportTimeout.evaluateCommand(self, cmd) + if SUCCESS != superResult: + return WARNINGS + if re.search('UNEXPECTED', cmd.logs['stdio'].getText()): + return WARNINGS + return SUCCESS + +class MozillaOSXReftest(MozillaReftest): + command = ["../../objdir/dist/Minefield.app/Contents/MacOS/firefox", + "-console", + "-P", + "default", + "-reftest", + "reftest.list"] + +class MozillaCrashtest(MozillaReftest): + name = "crashtest" + description = ["crashtest"] + descriptionDone = ["crashtest complete"] + +class MozillaOSXCrashtest(MozillaCrashtest): + command = ["../../objdir/dist/Minefield.app/Contents/MacOS/firefox", + "-console", + "-P", + "default", + "-reftest", + "crashtests.list"] + +class MozillaMochitest(ShellCommandReportTimeout): + name = "mochitest" + warnOnFailure = True + description = ["mochitest"] + descriptionDone = ["mochitest complete"] + command = ["python", + "runtests.py", + "--appname=../../../dist/bin/firefox", + "--autorun", + "--console-level=INFO", + "--close-when-done"] + + def createSummary(self, log): + passCount = 0 + failCount = 0 + todoCount = 0 + for line in log: + if "INFO Passed:" in line: + passCount = int(line.split()[-1]) + if "INFO Failed:" in line: + failCount = int(line.split()[-1]) + if "INFO Todo:" in line: + todoCount = int(line.split()[-1]) + summary = "TinderboxPrint: mochitest<br/>" + if not (passCount + failCount + todoCount): + summary += "FAIL\n" + else: + summary += str(passCount) + "/" + str(failCount) + "/" + str(todoCount) + "\n" + self.addCompleteLog('summary', summary) + + def evaluateCommand(self, cmd): + superResult = ShellCommandReportTimeout.evaluateCommand(self, cmd) + if SUCCESS != superResult: + return WARNINGS + if re.search('ERROR FAIL', cmd.logs['stdio'].getText()): + return WARNINGS + if re.search('ERROR TODO WORKED', cmd.logs['stdio'].getText()): + return WARNINGS + if re.search('FAIL Exited', cmd.logs['stdio'].getText()): + return WARNINGS + if not re.search('INFO PASS', cmd.logs['stdio'].getText()): + return WARNINGS + return SUCCESS + +class MozillaOSXMochitest(MozillaMochitest): + command = ["python", + "runtests.py", + "--appname=../../../objdir/dist/Minefield.app/Contents/MacOS/firefox", + "--autorun", + "--console-level=INFO", + "--close-when-done"] + +class MozillaMochichrome(ShellCommandReportTimeout): + name = "mochichrome" + warnOnFailure = True + description = ["mochichrome"] + descriptionDone = ["mochichrome complete"] + command = ["python", + "runtests.py", + "--appname=objdir/dist/bin/firefox", + "--chrome", + "--autorun", + "--console-level=INFO", + "--close-when-done"] + + def createSummary(self, log): + passCount = 0 + failCount = 0 + todoCount = 0 + for line in log: + if "INFO Passed:" in line: + passCount = int(line.split()[-1]) + if "INFO Failed:" in line: + failCount = int(line.split()[-1]) + if "INFO Todo:" in line: + todoCount = int(line.split()[-1]) + summary = "TinderboxPrint: chrome<br/>" + if not (passCount + failCount + todoCount): + summary += "FAIL\n" + else: + summary += str(passCount) + "/" + str(failCount) + "/" + str(todoCount) + "\n" + self.addCompleteLog('summary', summary) + + def evaluateCommand(self, cmd): + superResult = ShellCommandReportTimeout.evaluateCommand(self, cmd) + if SUCCESS != superResult: + return WARNINGS + if re.search('ERROR FAIL', cmd.logs['stdio'].getText()): + return WARNINGS + if re.search('FAIL Exited', cmd.logs['stdio'].getText()): + return WARNINGS + if not re.search('INFO PASS', cmd.logs['stdio'].getText()): + return WARNINGS + return SUCCESS + + +class MozillaOSXMochichrome(MozillaMochichrome): + command = ["python", + "runtests.py", + "--appname=../../../objdir/dist/Minefield.app/Contents/MacOS/firefox", + "--chrome", + "--autorun", + "--console-level=INFO", + "--close-when-done"] + +class MozillaBrowserChromeTest(ShellCommandReportTimeout): + name = "browser chrome test" + warnOnFailure = True + description = ["browser chrome test"] + descriptionDone = ["browser chrome test complete"] + command = ["python", + "runtests.py", + "--appname=../../../dist/bin/firefox", + "--autorun", + "--browser-chrome", + "--close-when-done"] + + def createSummary(self, log): + passCount = 0 + failCount = 0 + todoCount = 0 + for line in log: + if "Pass:" in line: + passCount = int(line.split()[-1]) + if "Fail:" in line: + failCount = int(line.split()[-1]) + if "Todo:" in line: + todoCount = int(line.split()[-1]) + summary = "TinderboxPrint: browser<br/>" + if not (passCount + failCount + todoCount): + summary += "FAIL\n" + else: + summary += str(passCount) + "/" + str(failCount) + "/" + str(todoCount) + "\n" + self.addCompleteLog('summary', summary) + + def evaluateCommand(self, cmd): + superResult = ShellCommandReportTimeout.evaluateCommand(self, cmd) + if SUCCESS != superResult: + return WARNINGS + if re.search('FAIL -', cmd.logs['stdio'].getText()): + return WARNINGS + if re.search('FAIL Exited', cmd.logs['stdio'].getText()): + return WARNINGS + return SUCCESS + +class MozillaOSXBrowserChromeTest(MozillaBrowserChromeTest): + command = ["python", + "runtests.py", + "--appname=../../../objdir/dist/Minefield.app/Contents/MacOS/firefox", + "--autorun", + "--browser-chrome", + "--close-when-done"] + +class MozillaA11YTest(MozillaMochichrome): + name = "a11y test" + warnOnFailure = True + description = ["a11y test"] + descriptionDone = ["a11y test complete"] + command = ["python", + "runtests.py", + "--appname=objdir/dist/bin/firefox", + "--console-level=INFO", + "--autorun", + "--a11y", + "--close-when-done"] + + def createSummary(self, log): + passCount = 0 + failCount = 0 + todoCount = 0 + for line in log: + if "INFO Passed:" in line: + passCount = int(line.split()[-1]) + if "INFO Failed:" in line: + failCount = int(line.split()[-1]) + if "INFO Todo:" in line: + todoCount = int(line.split()[-1]) + summary = "TinderboxPrint: a11y<br/>" + if not (passCount + failCount + todoCount): + summary += "FAIL\n" + else: + summary += str(passCount) + "/" + str(failCount) + "/" + str(todoCount) + "\n" + self.addCompleteLog('summary', summary) + + +class MozillaOSXA11YTest(MozillaA11YTest): + command = ["python", + "runtests.py", + "--appname=../../../dist/Minefield.app/Contents/MacOS/firefox", + "--a11y", + "--autorun", + "--console-level=INFO", + "--close-when-done"] + +class CreateProfile(ShellCommandReportTimeout): + name = "create profile" + warnOnFailure = True + description = ["create profile"] + descriptionDone = ["create profile complete"] + command = r'python testing/tools/profiles/createTestingProfile.py --clobber --binary objdir/dist/bin/firefox' diff -r 9e48e21dd4f6 master.cfg --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/master.cfg Sat Feb 07 12:21:02 2009 -0500 @@ -0,0 +1,238 @@ +# -*- python -*- +# ex: set syntax=python: + +# This is a sample buildmaster config file. It must be installed as +# 'master.cfg' in your buildmaster's base directory (although the filename +# can be changed with the --basedir option to 'mktap buildbot master'). + +# It has one job: define a dictionary named BuildmasterConfig. This +# dictionary has a variety of keys to control different aspects of the +# buildmaster. They are documented in docs/config.xhtml . + + +# This is the dictionary that the buildmaster pays attention to. We also use +# a shorter alias to save typing. +c = BuildmasterConfig = {} + +####### BUILDSLAVES + +# the 'slaves' list defines the set of allowable buildslaves. Each element is +# a tuple of bot-name and bot-password. These correspond to values given to +# the buildslave's mktap invocation. +from buildbot.buildslave import BuildSlave +#c['slaves'] = [BuildSlave("p6t-slave", "mozilla"), BuildSlave("macbook-slave", "mozilla"), BuildSlave("neb-slave", "mozilla")] +c['slaves'] = [BuildSlave("local-slave", "mozilla")] + +# to limit to two concurrent builds on a slave, use +# c['slaves'] = [BuildSlave("bot1name", "bot1passwd", max_builds=2)] + + +# 'slavePortnum' defines the TCP port to listen on. This must match the value +# configured into the buildslaves (with their --master option) + +c['slavePortnum'] = 9989 + +####### CHANGESOURCES + +# the 'change_source' setting tells the buildmaster how it should find out +# about source code changes. Any class which implements IChangeSource can be +# put here: there are several in buildbot/changes/*.py to choose from. + +#from buildbot.changes.pb import PBChangeSource +#c['change_source'] = [] +#c['change_source'].append(PBChangeSource()) + +# For example, if you had CVSToys installed on your repository, and your +# CVSROOT/freshcfg file had an entry like this: +#pb = ConfigurationSet([ +# (None, None, None, PBService(userpass=('foo', 'bar'), port=4519)), +# ]) + +# then you could use the following buildmaster Change Source to subscribe to +# the FreshCVS daemon and be notified on every commit: +# +#from buildbot.changes.freshcvs import FreshCVSSource +#fc_source = FreshCVSSource("cvs.example.com", 4519, "foo", "bar") +#c['change_source'] = fc_source + +# or, use a PBChangeSource, and then have your repository's commit script run +# 'buildbot sendchange', or use contrib/svn_buildbot.py, or +# contrib/arch_buildbot.py : +# +#from buildbot.changes.pb import PBChangeSource +#c['change_source'] = PBChangeSource() + + + +####### SCHEDULERS + +## configure the Schedulers + +from buildbot.scheduler import Scheduler +c['schedulers'] = [] +#c['schedulers'].append(Scheduler(name="all", branch=None, +# treeStableTimer=5, +# builderNames=["buildbot-f10", "buildbot-osx", "buildbot-xp"])) +# + +####### BUILDERS + +# the 'builders' list defines the Builders. Each one is configured with a +# dictionary, using the following keys: +# name (required): the name used to describe this bilder +# slavename (required): which slave to use, must appear in c['bots'] +# builddir (required): which subdirectory to run the builder in +# factory (required): a BuildFactory to define how the build is run +# periodicBuildTime (optional): if set, force a build every N seconds + +# buildbot/process/factory.py provides several BuildFactory classes you can +# start with, which implement build processes for common targets (GNU +# autoconf projects, CPAN perl modules, etc). The factory.BuildFactory is the +# base class, and is configured with a series of BuildSteps. When the build +# is run, the appropriate buildslave is told to execute each Step in turn. + +# the first BuildStep is typically responsible for obtaining a copy of the +# sources. There are source-obtaining Steps in buildbot/steps/source.py for +# CVS, SVN, and others. + +#John's Simple Factory +from buildbot.process import factory +from buildbot.steps.shell import ShellCommand, Compile +from buildbot.steps.source import Mercurial +f1 = factory.BuildFactory() +import os +f1.addStep(ShellCommand, name='current date-time', command='~/pytime.py') +f1.addStep(Mercurial, repourl="http://hg.mozilla.org/mozilla-central/", mode='update'); +f1.addStep(ShellCommand, flunkOnFail=False, haltOnFail=False, name='clean-obj', command='rm -rf obj',) +f1.addStep(Compile, name='build', command=['make', '-f', 'client.mk', 'build']) + + +## +## Mac OS X +## +from buildbot.mozbuild import * + +osxFactory = factory.BuildFactory() + +osxFactory.addStep(ShellCommand, name='current date-time', command='~/pytime.py') +osxFactory.addStep(ShellCommand, name="mozconfig contents", + command=["cat","/home/jhford/.mozconfig"]) +osxFactory.addStep(Mercurial, repourl="http://hg.mozilla.org/mozilla-central/", mode='update'); +osxFactory.addStep(ShellCommand, flunkOnFail=False, haltOnFail=False, name='clean-obj', command='rm -rf obj',) +osxFactory.addStep(Compile, name='build', command=['make', '-f', 'client.mk', 'build']) +osxFactory.addStep(ShellCommand, name='check', workdir='build/objdir', command='make -k check') +osxFactory.addStep(CreateProfile, + warnOnWarnings=True, + env=MozillaEnvironments['osx'], + clobber=True) +osxFactory.addStep(MozillaOSXReftest, warnOnWarnings=True, + workdir="build/layout/reftests", + env=MozillaEnvironments['osx']) +osxFactory.addStep(MozillaOSXCrashtest, warnOnWarnings=True, + workdir="build/testing/crashtest", + env=MozillaEnvironments['osx']) +#osxFactory.addStep(MozillaMochitest, warnOnWarnings=True, +# workdir="build/objdir/_tests/testing/mochitest", +# env=MozillaEnvironments['osx']) +#osxFactory.addStep(MozillaMochichrome, warnOnWarnings=True, +# workdir="build/objdir/_tests/testing/mochitest", +# env=MozillaEnvironments['osx']) +#osxFactory.addStep(MozillaBrowserChromeTest, warnOnWarnings=True, +# workdir="build/objdir/_tests/testing/mochitest", +# env=MozillaEnvironments['osx']) + + +b1 = {'name': "local-osx", + 'slavename': "local-slave", + 'builddir': "osx", + 'factory': f1, + } + +mozbuilder = {'name': "mozilla-osx", + 'slavename': "local-slave", + 'builddir': "moz-osx", + 'factory': osxFactory, + } + + +b2 = {'name': "buildbot-f10", + 'slavename': "p6t-slave", + 'builddir': "linux", + 'factory': f1, + } + +#b3 = {'name': "buildbot-xp", +# 'slavename': "neb-slave", +# 'builddir': "xp", +# 'factory': f1, +# } + + +c['builders'] = [] +c['builders'].append(b1) +c['builders'].append(mozbuilder) +#c['builders'].append(b2) +#c['builders'].append(b3) + + +####### STATUS TARGETS + +# 'status' is a list of Status Targets. The results of each build will be +# pushed to these targets. buildbot/status/*.py has a variety to choose from, +# including web pages, email senders, and IRC bots. + +c['status'] = [] + +from buildbot.status import html +c['status'].append(html.WebStatus(http_port=8010, allowForce=True)) + +from buildbot.status import mail +c['status'].append(mail.MailNotifier(fromaddr="buildbot@localhost", + extraRecipients=["buildlogs@johnford.info"], + sendToInterestedUsers=False)) + +from buildbot.status import words +c['status'].append(words.IRC(host="irc.mozilla.org", nick="buildbot-seneca", + channels=["#seneca-build"])) + +from buildbot.status import client +c['status'].append(client.PBListener(9988)) + + +####### DEBUGGING OPTIONS + +# if you set 'debugPassword', then you can connect to the buildmaster with +# the diagnostic tool in contrib/debugclient.py . From this tool, you can +# manually force builds and inject changes, which may be useful for testing +# your buildmaster without actually commiting changes to your repository (or +# before you have a functioning 'sources' set up). The debug tool uses the +# same port number as the slaves do: 'slavePortnum'. + +#c['debugPassword'] = "debugpassword" + +# if you set 'manhole', you can ssh into the buildmaster and get an +# interactive python shell, which may be useful for debugging buildbot +# internals. It is probably only useful for buildbot developers. You can also +# use an authorized_keys file, or plain telnet. +#from buildbot import manhole +#c['manhole'] = manhole.PasswordManhole("tcp:9999:interface=127.0.0.1", +# "admin", "password") + + +####### PROJECT IDENTITY + +# the 'projectName' string will be used to describe the project that this +# buildbot is working on. For example, it is used as the title of the +# waterfall HTML page. The 'projectURL' string will be used to provide a link +# from buildbot HTML pages to your project's home page. + +c['projectName'] = "Mozilla @ Seneca" +c['projectURL'] = "http://zenit.senecac.on.ca/wiki" + +# the 'buildbotURL' string should point to the location where the buildbot's +# internal web server (usually the html.Waterfall page) is visible. This +# typically uses the port number set in the Waterfall 'status' entry, but +# with an externally-visible host name which the buildbot cannot figure out +# without some help. + +c['buildbotURL'] = "http://localhost:8010/"