Difference between revisions of "User:Mdsouza/Assignment 01: Build the Fox"

From CDOT Wiki
Jump to: navigation, search
 
m (→‎The Build Begins)
 
(13 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
=Overview=
 
=Overview=
  
 +
The way I decided to proceed with this assignment was to read all the information out there about how the build process works before trying anything out. I did this to decrease my chances that I do not fall into a "try, fail, try again" scenario. However, as it came to be, finding a bug in software is inevitable. :(
  
 +
In the report below I mentioned the tools one needs, steps I took, and problems that I tackled to get me through the process of building firefox "minefield" using CVS!
 +
 +
Note: I initially tried to build firefox 1.5.0.7 without using CVS, but ran into way too many problems to document. I then moved on to the minefield version and the problems that spitted out were easier to fix.
  
 
=Computer Specs=
 
=Computer Specs=
Line 11: Line 15:
  
 
=Steps Took To Build the Fox=
 
=Steps Took To Build the Fox=
−
 
  
 
==Did The Boring Stuff==
 
==Did The Boring Stuff==
Line 22: Line 25:
 
*[http://developer.mozilla.org/en/docs/Mozilla_Build_FAQ Mozilla Build FAQ]
 
*[http://developer.mozilla.org/en/docs/Mozilla_Build_FAQ Mozilla Build FAQ]
 
*[http://cs.nyu.edu/rgrimm/teaching/fa05-oop/windows-make.html Getting Make to place nice with Cygwin]
 
*[http://cs.nyu.edu/rgrimm/teaching/fa05-oop/windows-make.html Getting Make to place nice with Cygwin]
−
* [http://forums.mozillazine.org/viewforum.php?f=23 Firefox Build Forum @ MozillaZine]
+
*[http://forums.mozillazine.org/viewforum.php?f=23 Firefox Build Forum @ MozillaZine]
 +
*[http://cs1.senecac.on.ca/wiki/index.php/Assignment_1_List Other Student's Assignments]
  
 
==Downloaded And Installed The Tools==
 
==Downloaded And Installed The Tools==
  
 +
*Microsoft® Visual Studio .NET 2003® (C++ module is needed mainly). It's available from [http://acs.senecac.on.ca/pages/download.php ACS] -- [http://sonic.senecac.on.ca/download/getfile.php?username=sfernan2&filename=MSDN/2072.2.ISO Pre-requesite Disc], [http://sonic.senecac.on.ca/download/getfile.php?username=sfernan2&filename=MSDN/2070.ISO Disc 1], [http://sonic.senecac.on.ca/download/getfile.php?username=sfernan2&filename=MSDN/2071.ISO Disc 2]
 +
*[http://www.rarlabs.com WinRar] to extract several archive files
 +
*[ftp://ftp.cygwin.com/pub/cygwin/setup.exe Cygwin] | [http://developer.mozilla.org/en/docs/Windows_Build_Prerequisites#GNU_Tools_for_Microsoft_Windows_.28Cygwin.29 More Info] | [http://www.cygwin.com Cygwin Main Website]
 +
 +
When installing Cygwin the following packages need to be selected:
 +
 +
<pre>
 +
* ash -- UNIX-like command line interpreter shell (Base category)
 +
* coreutils -- GNU core utilities (includes fileutils, install, sh-utils, and textutils) (Base category)
 +
* cvs -- concurrent versions system (Devel category)
 +
* diffutils -- file comparison utility (Base category)
 +
* findutils (Base category)
 +
* gawk -- pattern matching language (Base and Interpretors categories)
 +
* grep -- text search tool (Base category)
 +
* libiconv -- character set conversion (Devel category)
 +
* make -- dependency analyzer for software builds (Devel category) (not mandatory if the version is 3.81 -- we need 3.80)
 +
* patchutils -- a small collection of programs that operate on patch files (Devel category)
 +
* perl -- a scripting language used to control parts of the build (Interpreters category)
 +
* sed -- a search and replace language (Base category)
 +
* unzip -- zip file extraction (Archive category)
 +
* zip -- zip file creation (Archive category)
 +
</pre>
 +
 +
*[http://cygwin.paracoda.com/release/make/make-3.80-1.tar.bz2 Make 3.80] (to overwrite the 3.81 version from the Cygwin install)
 +
 +
Once '''MAKE''' is downloaded and saved on your hard drive:
 +
 +
<pre>
 +
1. Use WinRAR to extract the file to any folder of your choosing.
 +
2. Copy the make.exe file located within the usr/bin folder and overwrite the make.exe located in C:\cygwin\bin
 +
</pre>
 +
 +
*[http://ftp.mozilla.org/pub/mozilla.org/mozilla/libraries/win32/moztools-static.zip MozTools] | [http://developer.mozilla.org/en/docs/Windows_Build_Prerequisites#moztools More Info]
 +
 +
Once downloaded, extract the contents to '''C:\proj\moztools'''.
 +
 +
*[http://www.microsoft.com/downloads/details.aspx?FamilyID=0baf2b35-c656-4969-ace8-e4c0c0716adb&DisplayLang=en Microsoft ® Windows Server® 2003 R2 Platform SDK] | [http://developer.mozilla.org/en/docs/Windows_Build_Prerequisites#Platform_SDK More Info]
  
 +
Once downloaded, the "Windows Core SDK" (Tools, Build Environment, and Redistributable Components) and the "Web Workshop SDK" (Build Environment) must be installed. If you are using at least Visual Studio .NET 2003, you already have all the necessary SDKs for building mozilla and can skip this requirement step.
  
 
==The Build Begins==
 
==The Build Begins==
  
 +
I mainly followed the guide from [http://cs.senecac.on.ca/%7Edavid.humphrey/writing/firefox-win32-build.html Dave's site] but changed a few aspects of it so that it worked for the Visual Studio 2003 version and the way I setup my computer.
  
 +
1. Created a directory called '''C:\proj'''
 +
 +
2. [[User:Mdsouza/Assignment_01:_Build_the_Fox#Downloaded_And_Installed_The_Tools|Downloaded and Installed]] the necessary software above.
 +
 +
3. Created my [[User:Mdsouza/Assignment_01:_Build_the_Fox#Batch_File|batch file]] and [[User:Mdsouza/Assignment_01:_Build_the_Fox#.mozconfig_File|.mozconfig file]].
 +
 +
4. Navigated to the location of where I stored my batch file ('''C:\proj''') and double-clicked it to execute.
 +
 +
5. When the bash prompt came up, I typed the following commands...
 +
 +
Navigated to the '''C:\proj''' folder. Logged in ('''anonymous''' is the password):
 +
 +
<pre>
 +
cvs login
 +
</pre>
 +
 +
Navigated to the '''C:\proj''' folder. Checked out the build script:
 +
 +
<pre>
 +
cvs co mozilla/browser/config mozilla/client.mk
 +
</pre>
 +
 +
6. It created a directory called '''C:\proj\home\mozilla'''. I copied my .mozconfig file over to the '''C:\proj\home\mozilla''' folder.
 +
 +
7. Went back to the prompt and typed...
 +
 +
Navigated to the '''C:\proj\home\mozilla''' folder. Got the full source code:
 +
 +
<pre>
 +
make -f client.mk checkout
 +
</pre>
 +
 +
Built the code:
 +
 +
<pre>
 +
make -f client.mk build
 +
</pre>
 +
 +
8. Kept waiting for it to build (it took a couple of hours suprisingly). A [[User:Mdsouza/Assignment_01:_Build_the_Fox#Problems_And_Solutions|couple of bugs]] below came up, I checked my
 +
[[User:Mdsouza/Assignment_01:_Build_the_Fox#Did_The_Boring_Stuff|resources]] and [http://www.google.com Google], fixed the problems and FINALLY finished it successfully!
  
 
==Problems And Solutions==
 
==Problems And Solutions==
  
 +
===Problem 1===
  
 +
I originally did not have my enviromental variables correctly set up. So I got this error:
 +
 +
<pre>
 +
$ make -f client.mk build
 +
Adding client.mk options from /cygdrive/c/proj/mozilla/.mozconfig:
 +
    MOZ_OBJDIR=$(TOPSRCDIR)firefox-objdir
 +
make[1]: Entering directory `/cygdrive/c/proj/mozilla'
 +
cd /cygdrive/c/proj/mozillafirefox-objdir
 +
/cygdrive/c/proj/mozilla/configure
 +
Adding configure options from /cygdrive/c/proj/mozilla/.mozconfig:
 +
  --disable-static
 +
  --enable-shared
 +
  --disable-optimize
 +
  --enable-debug
 +
  --enable-canvas
 +
  --disable-installer
 +
creating cache ./config.cache
 +
checking host system type... i686-pc-cygwin
 +
checking target system type... i686-pc-cygwin
 +
checking build system type... i686-pc-cygwin
 +
checking for gawk... gawk
 +
checking for nsinstall... no
 +
checking for gcc... cl
 +
checking whether the C compiler (cl  ) works... no
 +
configure: error: installation or configuration problem: C compiler cannot create executables.
 +
*** Fix above errors and then restart with "make -f client.mk build"
 +
make[1]: *** [configure] Error 1
 +
make[1]: Leaving directory `/cygdrive/c/proj/mozilla'
 +
make: *** [/cygdrive/c/proj/mozillafirefox-objdir/Makefile] Error 2
 +
</pre>
 +
 +
I solved this problem so I went to the [http://developer.mozilla.org/en/docs/Mozilla_Build_FAQ#Win32-specific_questions Mozilla Build FAQ] and it told me to run vcvars32.bat, delete my config.cache file (in the mozilla or object directory) and then build again.
 +
 +
I tried this but it still failed. So then I created a batch file, included a call to setup several variables, and double-checked that all directories in the variables were correct. This fixed the problem.
 +
 +
===Problem 2===
 +
 +
The other problem was that even though I had 1GB of ram, and had a 1.5GB pagefile, it crashed my computer twice. The first time my page file reached from about 130mb (when I closed all the uncessary programs) to 909mb. The second time my page file reached 707mb and crashed. I got the following error to inform me about an hour into the build.
 +
 +
<pre>
 +
$ make -f client.mk build
 +
      8 [unknown (0x20EE0)] sed (134904) C:\cygwin\bin\sed.exe: *** fatal error - C:\
 +
cygwin\bin\sed.exe: *** CreateThread failed for sig - 0x0<0x0>, Win32 error 1450
 +
/bin/sh: /usr/bin/sed: Resource temporarily unavailable
 +
/bin/sh: /usr/bin/sed: Resource temporarily unavailable
 +
make: execvp: /bin/sh.exe: Resource temporarily unavailable
 +
make: execvp: /bin/sh.exe: Resource temporarily unavailable
 +
client.mk:482: /.mozconfig.mk: No such file or directory
 +
client.mk:483: /build/unix/modules.mk: No such file or directory
 +
make: execvp: /bin/sh.exe: Resource temporarily unavailable
 +
make: *** No rule to make target `/build/unix/modules.mk'.  Stop.
 +
</pre>
 +
 +
I researched the problem, but no solutions helped. I then defragged my hard drive, rebooted, closed all the extra programs) ran the build and it worked (thankfully)!
 +
 +
==Apendix==
 +
 +
 +
===Batch File===
 +
 +
<pre>
 +
rem --- Set HOME so that cvs and ssh work correctly
 +
rem --- cvs uses HOME to locate your .cvspass file, and ssh to locate your .ssh file
 +
rem --- if you are using ssh, your HOME should match the home directory specified in /etc/passwd.
 +
rem --- See http://www.cygwin.com/faq/faq0.html.
 +
set HOME=C:\proj\home
 +
 +
rem --- CVS Setup
 +
SET CVSROOT=:pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot
 +
SET CVS_RSH=ssh
 +
 +
rem --- Setup the paths to the moztools build libraries
 +
set MOZ_TOOLS=C:\proj\moztools
 +
set GLIB_PREFIX=%MOZ_TOOLS%
 +
set LIBIDL_PREFIX=%MOZ_TOOLS%
 +
 +
rem --- Scrub these variables first
 +
SET INCLUDE=
 +
SET LIB=
 +
SET PATH=C:\;C:\windows\system32;C:\windows\system32\wbem
 +
 +
rem --- Prepend cygwin
 +
SET PATH=C:\cygwin\bin;%PATH%
 +
 +
rem --- Setup VC7 compiler environment vars
 +
CALL "C:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\bin\vcvars32.bat"
 +
 +
rem --- Add glib/libidl to build environment
 +
SET PATH=%PATH%;%GLIB_PREFIX%;%GLIB_PREFIX%\bin
 +
SET INCLUDE=%GLIB_PREFIX%\include;%INCLUDE%
 +
SET LIB=%GLIB_PREFIX%\lib;%LIB%
 +
 +
rem --- moztools comes last after glib/libIDL
 +
SET PATH=%PATH%;%MOZ_TOOLS%\bin
 +
 +
rem --- Typically the last thing the script does is launch a cygwin shell
 +
rem --- watch for your ~/.profile and /etc/profile which may overwrite your carefully setup PATH!
 +
bash --login -i
 +
</pre>
 +
 +
===.mozconfig File===
 +
 +
<pre>
 +
# Building Firefox Trunk with Debugging
 +
. $topsrcdir/browser/config/mozconfig
 +
 +
# Put all obj files in one place, not in src tree
 +
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/firefox-objdir
 +
ac_add_options --disable-static
 +
ac_add_options --enable-shared
 +
 +
# Debug Build Setup Options
 +
ac_add_options --disable-optimize
 +
ac_add_options --enable-debug
 +
 +
# I'm using Canvas for my work
 +
ac_add_options --enable-canvas
 +
 +
# I disabled the installer
 +
ac_add_options --disable-installer
 +
</pre>
  
 
=Voila!=
 
=Voila!=
  
 +
[[Image: Mdsouza_a1.gif]]
 +
 +
=Conclusion=
  
 +
Like everyone who built firefox the first time, it is quite confusing, and needs a lot of patience to read all the documenation and fix the several bugs that one can encounter. This is the first time I did something as big as this, and this is also the first time I participated in an open source project (other than helping spread firefox).
  
−
=Conclusion=
+
Although this first experience was crazy I did like every minute of it (seriously!) and I hope to continue working on open source projects.

Latest revision as of 19:15, 20 September 2006

Overview

The way I decided to proceed with this assignment was to read all the information out there about how the build process works before trying anything out. I did this to decrease my chances that I do not fall into a "try, fail, try again" scenario. However, as it came to be, finding a bug in software is inevitable. :(

In the report below I mentioned the tools one needs, steps I took, and problems that I tackled to get me through the process of building firefox "minefield" using CVS!

Note: I initially tried to build firefox 1.5.0.7 without using CVS, but ran into way too many problems to document. I then moved on to the minefield version and the problems that spitted out were easier to fix.

Computer Specs

  • Intel Pentium IV 1.7 GHz
  • 1GB of RAM
  • 40GB Hard Drive
  • Windows XP Professional

Steps Took To Build the Fox

Did The Boring Stuff

Here are some of the sites I spent the most time reading:

Downloaded And Installed The Tools

When installing Cygwin the following packages need to be selected: 

* ash -- UNIX-like command line interpreter shell (Base category) 
* coreutils -- GNU core utilities (includes fileutils, install, sh-utils, and textutils) (Base category) 
* cvs -- concurrent versions system (Devel category) 
* diffutils -- file comparison utility (Base category) 
* findutils (Base category) 
* gawk -- pattern matching language (Base and Interpretors categories) 
* grep -- text search tool (Base category) 
* libiconv -- character set conversion (Devel category) 
* make -- dependency analyzer for software builds (Devel category) (not mandatory if the version is 3.81 -- we need 3.80)
* patchutils -- a small collection of programs that operate on patch files (Devel category) 
* perl -- a scripting language used to control parts of the build (Interpreters category) 
* sed -- a search and replace language (Base category) 
* unzip -- zip file extraction (Archive category) 
* zip -- zip file creation (Archive category)
  • Make 3.80 (to overwrite the 3.81 version from the Cygwin install)

Once MAKE is downloaded and saved on your hard drive: 

1. Use WinRAR to extract the file to any folder of your choosing.
2. Copy the make.exe file located within the usr/bin folder and overwrite the make.exe located in C:\cygwin\bin

Once downloaded, extract the contents to C:\proj\moztools.

Once downloaded, the "Windows Core SDK" (Tools, Build Environment, and Redistributable Components) and the "Web Workshop SDK" (Build Environment) must be installed. If you are using at least Visual Studio .NET 2003, you already have all the necessary SDKs for building mozilla and can skip this requirement step.

The Build Begins

I mainly followed the guide from Dave's site but changed a few aspects of it so that it worked for the Visual Studio 2003 version and the way I setup my computer.

1. Created a directory called C:\proj

2. Downloaded and Installed the necessary software above.

3. Created my batch file and .mozconfig file.

4. Navigated to the location of where I stored my batch file (C:\proj) and double-clicked it to execute.

5. When the bash prompt came up, I typed the following commands...

Navigated to the C:\proj folder. Logged in (anonymous is the password): 

cvs login

Navigated to the C:\proj folder. Checked out the build script: 

cvs co mozilla/browser/config mozilla/client.mk

6. It created a directory called C:\proj\home\mozilla. I copied my .mozconfig file over to the C:\proj\home\mozilla folder.

7. Went back to the prompt and typed...

Navigated to the C:\proj\home\mozilla folder. Got the full source code: 

make -f client.mk checkout

Built the code: 

make -f client.mk build

8. Kept waiting for it to build (it took a couple of hours suprisingly). A couple of bugs below came up, I checked my resources and Google, fixed the problems and FINALLY finished it successfully!

Problems And Solutions

Problem 1

I originally did not have my enviromental variables correctly set up. So I got this error: 

$ make -f client.mk build
Adding client.mk options from /cygdrive/c/proj/mozilla/.mozconfig:
    MOZ_OBJDIR=$(TOPSRCDIR)firefox-objdir
make[1]: Entering directory `/cygdrive/c/proj/mozilla'
cd /cygdrive/c/proj/mozillafirefox-objdir
/cygdrive/c/proj/mozilla/configure
Adding configure options from /cygdrive/c/proj/mozilla/.mozconfig:
  --disable-static
  --enable-shared
  --disable-optimize
  --enable-debug
  --enable-canvas
  --disable-installer
creating cache ./config.cache
checking host system type... i686-pc-cygwin
checking target system type... i686-pc-cygwin
checking build system type... i686-pc-cygwin
checking for gawk... gawk
checking for nsinstall... no
checking for gcc... cl
checking whether the C compiler (cl  ) works... no
configure: error: installation or configuration problem: C compiler cannot create executables.
*** Fix above errors and then restart with "make -f client.mk build"
make[1]: *** [configure] Error 1
make[1]: Leaving directory `/cygdrive/c/proj/mozilla'
make: *** [/cygdrive/c/proj/mozillafirefox-objdir/Makefile] Error 2

I solved this problem so I went to the Mozilla Build FAQ and it told me to run vcvars32.bat, delete my config.cache file (in the mozilla or object directory) and then build again.

I tried this but it still failed. So then I created a batch file, included a call to setup several variables, and double-checked that all directories in the variables were correct. This fixed the problem.

Problem 2

The other problem was that even though I had 1GB of ram, and had a 1.5GB pagefile, it crashed my computer twice. The first time my page file reached from about 130mb (when I closed all the uncessary programs) to 909mb. The second time my page file reached 707mb and crashed. I got the following error to inform me about an hour into the build. 

$ make -f client.mk build
      8 [unknown (0x20EE0)] sed (134904) C:\cygwin\bin\sed.exe: *** fatal error - C:\
cygwin\bin\sed.exe: *** CreateThread failed for sig - 0x0<0x0>, Win32 error 1450
/bin/sh: /usr/bin/sed: Resource temporarily unavailable
/bin/sh: /usr/bin/sed: Resource temporarily unavailable
make: execvp: /bin/sh.exe: Resource temporarily unavailable
make: execvp: /bin/sh.exe: Resource temporarily unavailable
client.mk:482: /.mozconfig.mk: No such file or directory
client.mk:483: /build/unix/modules.mk: No such file or directory
make: execvp: /bin/sh.exe: Resource temporarily unavailable
make: *** No rule to make target `/build/unix/modules.mk'.  Stop.

I researched the problem, but no solutions helped. I then defragged my hard drive, rebooted, closed all the extra programs) ran the build and it worked (thankfully)!

Apendix

Batch File

rem --- Set HOME so that cvs and ssh work correctly
rem --- cvs uses HOME to locate your .cvspass file, and ssh to locate your .ssh file
rem --- if you are using ssh, your HOME should match the home directory specified in /etc/passwd. 
rem --- See http://www.cygwin.com/faq/faq0.html.
set HOME=C:\proj\home

rem --- CVS Setup
SET CVSROOT=:pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot
SET CVS_RSH=ssh

rem --- Setup the paths to the moztools build libraries
set MOZ_TOOLS=C:\proj\moztools
set GLIB_PREFIX=%MOZ_TOOLS%
set LIBIDL_PREFIX=%MOZ_TOOLS%

rem --- Scrub these variables first
SET INCLUDE=
SET LIB=
SET PATH=C:\;C:\windows\system32;C:\windows\system32\wbem

rem --- Prepend cygwin
SET PATH=C:\cygwin\bin;%PATH%

rem --- Setup VC7 compiler environment vars
CALL "C:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\bin\vcvars32.bat"

rem --- Add glib/libidl to build environment
SET PATH=%PATH%;%GLIB_PREFIX%;%GLIB_PREFIX%\bin
SET INCLUDE=%GLIB_PREFIX%\include;%INCLUDE%
SET LIB=%GLIB_PREFIX%\lib;%LIB%

rem --- moztools comes last after glib/libIDL
SET PATH=%PATH%;%MOZ_TOOLS%\bin

rem --- Typically the last thing the script does is launch a cygwin shell
rem --- watch for your ~/.profile and /etc/profile which may overwrite your carefully setup PATH!
bash --login -i

.mozconfig File

# Building Firefox Trunk with Debugging
. $topsrcdir/browser/config/mozconfig

# Put all obj files in one place, not in src tree
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/firefox-objdir
ac_add_options --disable-static
ac_add_options --enable-shared

# Debug Build Setup Options
ac_add_options --disable-optimize
ac_add_options --enable-debug

# I'm using Canvas for my work
ac_add_options --enable-canvas

# I disabled the installer
ac_add_options --disable-installer

Voila!

Mdsouza a1.gif

Conclusion

Like everyone who built firefox the first time, it is quite confusing, and needs a lot of patience to read all the documenation and fix the several bugs that one can encounter. This is the first time I did something as big as this, and this is also the first time I participated in an open source project (other than helping spread firefox).

Although this first experience was crazy I did like every minute of it (seriously!) and I hope to continue working on open source projects.

Retrieved from "https://wiki.cdot.senecapolytechnic.ca/w/index.php?title=User:Mdsouza/Assignment_01:_Build_the_Fox&oldid=3050"