User:Mdsouza/Assignment 01: Build the Fox

Overview

The way I decided to proceed with this assignment was to read all the information out there about how the build process works before trying anything out. I did this to decrease my chances that I do not fall into a "try, fail, try again" scenario. However, as it came to be, finding a bug in software is inevitable. :(

In the report below I mentioned the tools one needs, steps I took, and problems that I tackled to get me through the process of building firefox "minefield" using CVS!

Note: I initially tried to build firefox 1.5.0.7 without using CVS, but ran into way too many problems to document. I then moved on to the minefield version and the problems that spitted out were easier to fix.

Computer Specs

• Intel Pentium IV 1.7 GHz
• 1GB of RAM
• 40GB Hard Drive
• Windows XP Professional

Steps Took To Build the Fox

Did The Boring Stuff

Here are some of the sites I spent the most time reading:

• Windows Build Prerequisites
• Windows Build Documentation
• David Humphrey's Guide to Building Firefox (and the links from there)
• Mozilla Build FAQ
• Getting Make to place nice with Cygwin
• Firefox Build Forum @ MozillaZine
• Other Student's Assignments

Downloaded And Installed The Tools

• Microsoft® Visual Studio .NET 2003® (C++ module is needed mainly). It's available from ACS -- Pre-requesite Disc, Disc 1, Disc 2
• WinRar to extract several archive files
• Cygwin | More Info | Cygwin Main Website

When installing Cygwin the following packages need to be selected:

* ash -- UNIX-like command line interpreter shell (Base category) 
* coreutils -- GNU core utilities (includes fileutils, install, sh-utils, and textutils) (Base category) 
* cvs -- concurrent versions system (Devel category) 
* diffutils -- file comparison utility (Base category) 
* findutils (Base category) 
* gawk -- pattern matching language (Base and Interpretors categories) 
* grep -- text search tool (Base category) 
* libiconv -- character set conversion (Devel category) 
* make -- dependency analyzer for software builds (Devel category) (not mandatory if the version is 3.81 -- we need 3.80)
* patchutils -- a small collection of programs that operate on patch files (Devel category) 
* perl -- a scripting language used to control parts of the build (Interpreters category) 
* sed -- a search and replace language (Base category) 
* unzip -- zip file extraction (Archive category) 
* zip -- zip file creation (Archive category)

• Make 3.80 (to overwrite the 3.81 version from the Cygwin install)

Once MAKE is downloaded and saved on your hard drive:

1. Use WinRAR to extract the file to any folder of your choosing.
2. Copy the make.exe file located within the usr/bin folder and overwrite the make.exe located in C:\cygwin\bin 

• MozTools | More Info

Once downloaded, extract the contents to C:\proj\moztools.

• Microsoft ® Windows Server® 2003 R2 Platform SDK | More Info

Once downloaded, the "Windows Core SDK" (Tools, Build Environment, and Redistributable Components) and the "Web Workshop SDK" (Build Environment) must be installed. If you are using at least Visual Studio .NET 2003, you already have all the necessary SDKs for building mozilla and can skip this requirement step.

The Build Begins

I mainly followed the guide from Dave's site but changed a few aspects of it so that it worked for the Visual Studio 2003 version and the way I setup my computer.

1. Created a directory called C:\proj

2. User:Mdsouza/Assignment_01:_Build_the_Fox#Downloaded_And_Installed_The_Tools Downloaded and Installed the necessary software above.

3. Created my batch file and .mozconfig file.

4. Navigated to the location of where I stored my batch file (C:\proj) and double-clicked it to execute.

5. When the bash prompt came up, I typed the following commands...

Navigated to the C:\proj folder. Logged in (anonymous is the password):

cvs login

Navigated to the C:\proj folder. Checked out the build script:

cvs co mozilla/browser/config mozilla/client.mk

6. It created a directory called C:\proj\home\mozilla. I copied my .mozconfig file over to the C:\proj\home\mozilla folder.

7. Went back to the prompt and typed...

Navigated to the C:\proj\home\mozilla folder. Got the full source code:

make -f client.mk checkout

Built the code:

make -f client.mk build

8. Kept waiting for it to build (it took a couple of hours suprisingly). A couple of bugs below came up, I checked my resources and Google, fixed the problems and FINALLY finished it successfully!

Problems And Solutions

Problem 1

I originally did not have my enviromental variables correctly set up. So I got this error:

$ make -f client.mk build
Adding client.mk options from /cygdrive/c/proj/mozilla/.mozconfig:
    MOZ_OBJDIR=$(TOPSRCDIR)firefox-objdir
make[1]: Entering directory `/cygdrive/c/proj/mozilla'
cd /cygdrive/c/proj/mozillafirefox-objdir
/cygdrive/c/proj/mozilla/configure
Adding configure options from /cygdrive/c/proj/mozilla/.mozconfig:
  --disable-static
  --enable-shared
  --disable-optimize
  --enable-debug
  --enable-canvas
  --disable-installer
creating cache ./config.cache
checking host system type... i686-pc-cygwin
checking target system type... i686-pc-cygwin
checking build system type... i686-pc-cygwin
checking for gawk... gawk
checking for nsinstall... no
checking for gcc... cl
checking whether the C compiler (cl  ) works... no
configure: error: installation or configuration problem: C compiler cannot create executables.
*** Fix above errors and then restart with "make -f client.mk build"
make[1]: *** [configure] Error 1
make[1]: Leaving directory `/cygdrive/c/proj/mozilla'
make: *** [/cygdrive/c/proj/mozillafirefox-objdir/Makefile] Error 2

I solved this problem so I went to the Mozilla Build FAQ and it told me to run vcvars32.bat, delete my config.cache file (in the mozilla or object directory) and then build again.

I tried this but it still failed. So then I created a batch file, included a call to setup several variables, and double-checked that all directories in the variables were correct. This fixed the problem.

Problem 2

The other problem was that even though I had 1GB of ram, and had a 1.5GB pagefile, it crashed my computer twice. The first time my page file reached from about 130mb (when I closed all the uncessary programs) to 909mb. The second time my page file reached 707mb and crashed. I got the following error to inform me about an hour into the build.

$ make -f client.mk build
      8 [unknown (0x20EE0)] sed (134904) C:\cygwin\bin\sed.exe: *** fatal error - C:\
cygwin\bin\sed.exe: *** CreateThread failed for sig - 0x0<0x0>, Win32 error 1450
/bin/sh: /usr/bin/sed: Resource temporarily unavailable
/bin/sh: /usr/bin/sed: Resource temporarily unavailable
make: execvp: /bin/sh.exe: Resource temporarily unavailable
make: execvp: /bin/sh.exe: Resource temporarily unavailable
client.mk:482: /.mozconfig.mk: No such file or directory
client.mk:483: /build/unix/modules.mk: No such file or directory
make: execvp: /bin/sh.exe: Resource temporarily unavailable
make: *** No rule to make target `/build/unix/modules.mk'.  Stop.

I researched the problem, but no solutions helped. I then defragged my hard drive, rebooted, closed all the extra programs) ran the build and it worked (thankfully)!

Apendix

Batch File

rem --- Set HOME so that cvs and ssh work correctly
rem --- cvs uses HOME to locate your .cvspass file, and ssh to locate your .ssh file
rem --- if you are using ssh, your HOME should match the home directory specified in /etc/passwd. 
rem --- See http://www.cygwin.com/faq/faq0.html.
set HOME=C:\proj\home

rem --- CVS Setup
SET CVSROOT=:pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot
SET CVS_RSH=ssh

rem --- Setup the paths to the moztools build libraries
set MOZ_TOOLS=C:\proj\moztools
set GLIB_PREFIX=%MOZ_TOOLS%
set LIBIDL_PREFIX=%MOZ_TOOLS%

rem --- Scrub these variables first
SET INCLUDE=
SET LIB=
SET PATH=C:\;C:\windows\system32;C:\windows\system32\wbem

rem --- Prepend cygwin
SET PATH=C:\cygwin\bin;%PATH%

rem --- Setup VC7 compiler environment vars
CALL "C:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\bin\vcvars32.bat"

rem --- Add glib/libidl to build environment
SET PATH=%PATH%;%GLIB_PREFIX%;%GLIB_PREFIX%\bin
SET INCLUDE=%GLIB_PREFIX%\include;%INCLUDE%
SET LIB=%GLIB_PREFIX%\lib;%LIB%

rem --- moztools comes last after glib/libIDL
SET PATH=%PATH%;%MOZ_TOOLS%\bin

rem --- Typically the last thing the script does is launch a cygwin shell
rem --- watch for your ~/.profile and /etc/profile which may overwrite your carefully setup PATH!
bash --login -i

.mozconfig File

# Building Firefox Trunk with Debugging
. $topsrcdir/browser/config/mozconfig

# Put all obj files in one place, not in src tree
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/firefox-objdir
ac_add_options --disable-static
ac_add_options --enable-shared

# Debug Build Setup Options
ac_add_options --disable-optimize
ac_add_options --enable-debug

# I'm using Canvas for my work
ac_add_options --enable-canvas

# I disabled the installer
ac_add_options --disable-installer

Voila!

Conclusion

Like everyone who built firefox the first time, it is quite confusing, and needs a lot of patience to read all the documenation and fix the several bugs that one can encounter. This is the first time I did something as big as this, and this is also the first time I participated in an open source project (other than helping spread firefox).

Although this first experience was crazy I did like every minute of it (seriously!) and I hope to continue working on open source projects.