How the Build Works

Talk Details

• Date: Sept 20, 2006
• Speaker: J. Paul Reed

Media

• Part I - Main Talk (streaming)
• Part II - Question/Answer (streaming)
• Part II - Question/Answer more (streaming)
• MP3 (91 MB)
• Slides

Talk Outline

Time	Topic	Details
00:00:13	Introducing J. Paul Reed	Professor David Humphrey of Seneca College introduces J. Paul Reed, Build Engineer, Mozilla Corporation.
00:02:00	Lecture Agenda	Overview of the history of the build system.
00:03:00	"You Have the Wrong Guy"	Paul that he releases the software and considers himself more of a release engineer. You should be contacting the real build engineers Benjamin Smedberg, Mark Mentovai, Chris Seawood, and Bryan Ryner.
00:03:47	Paul's Background	Paul sheds some light on his background with the Netscape Build team (1998), his work on VM's at VMWare as release engineer and how he came full circle back to the project as a Moco (Mozilla Corporation) employee.
00:04:11	History of the build system	Paul mentions how Netscape had a collection of make files. Supporting 20 Unix platforms, Win16 & Win32. The process was very mystical and brittle. Documentation on the process was very crude.
00:05:50	Code Rush Documentary	Paul mentions the PBS documentary on the challenges of having three outsiders build Mozilla on the 3 platforms (Windows, Unix and Macintosh).
00:06:58	Autotools	On the limited use of autoconf and lack of usage of automake and other utilities. Paul also elaborates on the autoconf systems usage of .in files and how they drive/fit into the build process.
00:07:50	No Autoconf?	Paul explains Why don't we need autoconf to build Mozilla? An old machine at Mozilla monitors code checked (specifically config.in which is the source config file) in an automatically do this.
00:08:46	Major Organs of the build system Explained.	Paul discusses Netscape Portable Runtime (NSPR), Netscape Security Service (NSS), C-LDAP SDK, JavaScript, Supporting Libraries and most importantly the actual Mozilla products (Firefox, Thunderbird, Sunbird etc.) All the above are in top level citizens, you get them whether or not you want them when you check out the code.
00:10:24	client.mk - Useful targets	checkout - grabs checkout of the source code (done this way so they can version the different modules - a feature that cvs does not support.) build - builds the application configure - run automatically when build is run fast-update - checks Bonsai and does an update and retrieves only the parts you are working on (unlike cvs update).
00:13:07	.mozconfig	Paul explains the importance of the .mozconfig file in the build process, and mk_add_option vs ac_add_options. Build Configurator is here to help (http://webtools.mozilla.org/build/config.cgi)
00:15:43	Interesting things about build configuration.	The system does not guess. If the compile fails it knows it could not find it. If the compiler knows where to find it and build system does not its still OK.
00:19:00	10 Lines	The 10 most important lines in build process. The entire process for browser, Thunderbird etc. is driven by these 10 lines
00:19:54	Explanation of the Tiers	Paul discusses the various tiers in the build system and their contents.
00:23:22	Two Pass Build System Explained.	Paul elaborates on the export and libs pass (and sometimes also a thrid pass called deps). The Export pass copies all the .h files for all modules. The libs pass creates libraries that are needed for other modules. The Deps pass handles the automatic dependencies for files that you have edited etc. He stressed that all of this is still driven by the rules.mk file.
00:25:00	Explanation of the rules.mk file.	Good news: You need not necessarily worry about it. Just know what the definitions are if you need to use them. Paul also explains that the reason the makefile so short it is driven by the rules.mk file.
00:28:09	More rules.mk variables	Paul explains you can provide the build system will the all the information in needs to create your files. Chrome is stored in .jar files. The build system can handle .jars on their own, there need not be any rules. EXTRA_PP_COMPONENTS similar to C pre-processor defines in C.
00:30:47	MDC	Paul mentions that MDC has some great documentation on build flags and other build documentation.
00:33:43	Local vs. Official Builds	Paul explains the difference between a local build and the official ones from Mozilla, i.e. -d build ID (datestring), post build packaging commencing - for official builds it uploads to ftp mirrors (unlike local builds which just allow you to run them), talk back reports sent back to Mozilla.
00:35:36	Interesting parts of the Mozilla Build system.	Paul explains how the build system "cruft" and why it seems "weird". He discusses the reasons behind this. Specifically the use of autoconf as a thin wrapper as well as the issues accompanying use of top level directories in CVS.
00:38:23	The system is not as self-contained as it should be.	Currently localization packaging updates and generation are part of Tinderbox right now but this is not where it should be. The right place for them would be the build system.
00:39:22	Re-inventing the wheel	Paul discusses why Mozilla has taken a not-invented here attitude. In essence he explains why they have re-written things already in existence on most modern systems.
00:40:40	Cross Platform Development	Paul discusses the remarkable fact that the browser can run on all three platforms without limited use of #ifdef pre-processor statements. He remarked on the use of the NSPR and its benefits. Other oddities discussed include why C++ exceptions,templates and the 33 other portability rules (http://www.mozilla.org/hacking/portable-cpp.html#portability_rules)
00:44:50	Futures of the Build system	Paul discusses where the build system may be headed: Scons, Componentized builds, Moving to SVN
00:48:18	Re-write the whole thing.	Benjamin Smedberg threatens to re-write large (if not the whole) build system as the build engineers are starting to agree it may be easier to do that then to add stuff to it.
00:48:38	Useful notes for new developers	-DDEBUG-username useful for checking specific stuff when you yourself are debugging (can be turned off when you are done debugging). --No remote -p prevents your existing Firefox profile from being corrupted when you build a custom version.
00:50:16	Helpful Tools	Paul discussed the use of other tools developers use and which would be useful to new entrants to the Firefox community. For example, MDC, tinderbox, LXR, Planet etc.
00:51:06	Obtaining Help	Paul explains that #build is meant for official builds and to talk to build engineers. They like to keep a high signal to roise For general questions and inquiries use #developers or #Seneca. However, he stressed that it is important not to hesitate to ask for help.