Open main menu

CDOT Wiki β

How the Build Works

Talk Details

  • Date: Sept 20, 2006
  • Speaker: J. Paul Reed, Mozilla Corporation
  • License: CC-BY-SA

Media

Talk Outline

TimeTopicDetails
00:00:13Introducing J. Paul ReedProfessor David Humphrey of Seneca College introduces J. Paul Reed, Build Engineer, Mozilla Corporation.
00:02:00Lecture Agenda Overview of the history of the build system.
00:03:00"You Have the Wrong Guy"Paul that he releases the software and considers himself more of a release engineer. You should be contacting the real build engineers Benjamin Smedberg, Mark Mentovai, Chris Seawood, and Bryan Ryner.
00:03:47Paul's BackgroundPaul sheds some light on his background with the Netscape Build team (1998), his work on VM's at VMWare as release engineer and how he came full circle back to the project as a Moco (Mozilla Corporation) employee.
00:04:11History of the build systemPaul mentions how Netscape had a collection of make files. Supporting 20 Unix platforms, Win16 & Win32. The process was very mystical and brittle. Documentation on the process was very crude.
00:05:50Code Rush DocumentaryPaul mentions the PBS documentary on the challenges of having three outsiders build Mozilla on the 3 platforms (Windows, Unix and Macintosh).
00:06:58AutotoolsOn the limited use of autoconf and lack of usage of automake and other utilities. Paul also elaborates on the autoconf systems usage of .in files and how they drive/fit into the build process.
00:07:50No Autoconf?Paul explains Why don't we need autoconf to build Mozilla? An old machine at Mozilla monitors code checked (specifically config.in which is the source config file) in an automatically do this.
00:08:46Major Organs of the build system Explained.Paul discusses Netscape Portable Runtime (NSPR), Netscape Security Service (NSS), C-LDAP SDK, JavaScript, Supporting Libraries and most importantly the actual Mozilla products (Firefox, Thunderbird, Sunbird etc.) All the above are in top level citizens, you get them whether or not you want them when you check out the code.
00:10:24client.mk - Useful targets
  • checkout - grabs checkout of the source code (done this way so they can version the different modules - a feature that cvs does not support.)
  • build - builds the application
  • configure - run automatically when build is run
  • fast-update - checks Bonsai and does an update and retrieves only the parts you are working on (unlike cvs update).
00:13:07.mozconfigPaul explains the importance of the .mozconfig file in the build process, and mk_add_option vs ac_add_options. Build Configurator is here to help (http://webtools.mozilla.org/build/config.cgi)
00:15:43Interesting things about build configuration.

The system does not guess. If the compile fails it knows it could not find it.

If the compiler knows where to find it and build system does not its still OK.
00:19:0010 LinesThe 10 most important lines in build process. The entire process for browser, Thunderbird etc. is driven by these 10 lines
00:19:54Explanation of the TiersPaul discusses the various tiers in the build system and their contents.
00:23:22Two Pass Build System Explained.Paul elaborates on the export and libs pass (and sometimes also a thrid pass called deps). The Export pass copies all the .h files for all modules. The libs pass creates libraries that are needed for other modules. The Deps pass handles the automatic dependencies for files that you have edited etc. He stressed that all of this is still driven by the rules.mk file.
00:25:00Explanation of the rules.mk file.Good news: You need not necessarily worry about it. Just know what the definitions are if you need to use them. Paul also explains that the reason the makefile so short it is driven by the rules.mk file.
00:28:09More rules.mk variablesPaul explains you can provide the build system will the all the information in needs to create your files. Chrome is stored in .jar files. The build system can handle .jars on their own, there need not be any rules. EXTRA_PP_COMPONENTS similar to C pre-processor defines in C.
00:30:47MDCPaul mentions that MDC has some great documentation on build flags and other build documentation.
00:33:43Local vs. Official BuildsPaul explains the difference between a local build and the official ones from Mozilla, i.e. -d build ID (datestring), post build packaging commencing - for official builds it uploads to ftp mirrors (unlike local builds which just allow you to run them), talk back reports sent back to Mozilla.
00:35:36Interesting parts of the Mozilla Build system.Paul explains how the build system "cruft" and why it seems "weird". He discusses the reasons behind this. Specifically the use of autoconf as a thin wrapper as well as the issues accompanying use of top level directories in CVS.
00:38:23The system is not as self-contained as it should be.Currently localization packaging updates and generation are part of Tinderbox right now but this is not where it should be. The right place for them would be the build system.
00:39:22Re-inventing the wheelPaul discusses why Mozilla has taken a not-invented here attitude. In essence he explains why they have re-written things already in existence on most modern systems.
00:40:40Cross Platform DevelopmentPaul discusses the remarkable fact that the browser can run on all three platforms without limited use of #ifdef pre-processor statements. He remarked on the use of the NSPR and its benefits. Other oddities discussed include why C++ exceptions,templates and the 33 other portability rules (http://www.mozilla.org/hacking/portable-cpp.html#portability_rules)
00:44:50Futures of the Build systemPaul discusses where the build system may be headed: Scons, Componentized builds, Moving to SVN
00:48:18Re-write the whole thing.Benjamin Smedberg threatens to re-write large (if not the whole) build system as the build engineers are starting to agree it may be easier to do that then to add stuff to it.
00:48:38Useful notes for new developers-DDEBUG-username useful for checking specific stuff when you yourself are debugging (can be turned off when you are done debugging). --No remote -p prevents your existing Firefox profile from being corrupted when you build a custom version.
00:50:16Helpful ToolsPaul discussed the use of other tools developers use and which would be useful to new entrants to the Firefox community. For example, MDC, tinderbox, LXR, Planet etc.
00:51:06Obtaining HelpPaul explains that #build is meant for official builds and to talk to build engineers. They like to keep a high signal to noise For general questions and inquiries use #developers or #Seneca. However, he stressed that it is important not to hesitate to ask for help.