Mozharness

Project Name

Mozharness

Project Description

Imagine that we did not have to touch the Mozilla buildbot factories but instead we maintained a bunch of script for all the different jobs they run?

It would be good if we could create scripts that told a machine how to generate an optimized build, a debug build, unit tests, talos runs, locale repackages. If you look in the tools/scripts repo you can see that we have a simple shell file to do this for the fuzzing automation. The buildbot factory that calls it is called mozharness and it is very simple.

mozharness is a single factory that buildbot can run; within mozharness, a script is selected for a specific job. This moves the job-definition responsibility out from buildbot; by separating the pieces, you avoid excessive complexity in buildbot and also open up the possibility of running mozharness manually (e.g., by a developer, to test something)

Initial contacts: Armenzg

Project Leader(s)

adrianp: Adrian Pruteanu

mustafaj: Mustafa Redha

Project Contributor(s)

Armenzg

Project Details

• Objective: Improve efficiency while lowering the entry level knowledge required from new developers to get involved

• http://hg.mozilla.org/users/asasaki_mozilla.com/mozharness

• requirements: pylint and coverage

• ./unit.sh to run unit test coverage

• scripts directory contain scripts

• documentation is required

• API is changing - WIP in https://bugzilla.mozilla.org/show_bug.cgi?id=574473

• scripts written in python

• separate framework - which buildbot calls

 What are we building?

• Profile Build

• Debug Build - For OS (Mac, Linux, Windows, 32/64 etc)

• Packaged Test - contains test for the build

• Tests - Mochitests, Reftests, xpcshell, ... 10 in total
  • l10n builds

  The problem that mozharness solves:

• buildbot uses a "factory" to run a job

• there are (becoming) too many different "factories" within buildbot

Project Plan

Goals for each release:

adrianp

mustafaj

0.1 - Create the MercurialVCS class that is responsible for basic hg operations (clone, pull, push) each with an individual method 0.2 - Implement all hgtool actions as individual methods in MercurialVCS. At this point MercurialVCS() should be importable (useful) for Mozharness - Document usage for programmers (inheritance, import, whatever else not pertaining to users shell interaction) 0.3 - lib/source/mercurial.py should be callable via shell with parameters matching each method (ie: ./mercurial.py checkout [src]?) - The Class should be generalized to ensure portability to other D/VCSs (git, svn, cvs)

0.1 0.2 0.3

Project News

• Friday October 8, 2010:
  • I will try to contact Armen today and find out what his schedule is like, and coordinate with everybody to setup a meeting in CDOT conference room. -Mustafa

• Tuesday October 12, 2010:
  • Adrian made initial Contact with Armen, meeting set for 3pm Wendesday October 13, 2010 -Mustafa

• Wednesday October 13, 2010:
  • The mozharness/buildapi/releasebugs group met today with Armen and Ctyler in a conference call. The conference call is logged below -Mustafa

First meeting: mozharness call (2010-10-12 15:00 EDT)

1. What is mozharness (Armen)
2. Initial direction
3. Q&A

Atendees:

• ctyler, mustafaj, armenzg, adrianp, asingh

Blogs:

• Armen: http://armenzg.blogspot.com/
• Mustafa: http://mustafaredha.wordpress.com/
• adrianp: http://blog.esmnetworks.com/

... all of which appear on the CDOT Planet