Difference between revisions of "Profile the build system"

From CDOT Wiki
Jump to: navigation, search
(→‎Project Data: Added build tests on laptop)
(→‎Project Contributor(s))
Line 23: Line 23:
 
* [[User:AaronMT | Aaron Train]]
 
* [[User:AaronMT | Aaron Train]]
 
* [[User:cbishop2 | Chris Bishop]]
 
* [[User:cbishop2 | Chris Bishop]]
 +
* [[User:cdpatel1 | Chinmay Patel]]
 +
* [[User:Evmarbella | Ezadkiel V. Marbella]]
 +
* [[User:Scott | Scott Lunel]]
  
 
== Project Details ==
 
== Project Details ==

Revision as of 13:46, 21 September 2008

Name of Project

Profile the build system

Description

It's a well known fact that it takes longer to build on Windows than on a comparable Linux or Mac machine. We don't know exactly why, however. There have been many theories, but no real data. Profiling the build system would allow us to figure out where all of the time is being spent. The best place to start might be by adding some profiling to GNU Make, to figure out which targets in our makefiles take the most time. From there, depending on the results, profiling could be added to other parts of the build system to narrow down specific bottlenecks.

References:

  1. http://wiki.services.openoffice.org/wiki/BuildSpeedup - OpenOffice.org did something similar

Resources: ted

Leader(s)

Project Contributor(s)

Project Details

  • Collect Data
    • Get build time info from others
      • Build From Scratch, Rebuild with no changes, using/not using -j4 option, hardware info (RAM, CPU, RPM)


  • The -j4 option is used in the .mozconfig file
  • -j specifies how many concurrent jobs you can run, so for some parts of the tree you get more makes going at once
    • mk_add_options MOZ_MAKE_FLAGS=-j4

Project Data

  • Please feel free to fill in the table to help me collect more data
  • YOU WILL BE CONSIDERED AS A PROJECT CONTRIBUTOR FOR PROVIDING DATA
  • you can time your build by adding the "time" command in front of your build command like so:
    • time make -f client.mk build
</pre> </pre> </pre>
Build Time Data
Build From Scratch

HH:MM:SS

Rebuilt No Changes

HH:MM:SS

with -j# RAM

(GB)

CPU

(GHz)

HD

(rpm)

OS User Mozconfig
0:55:05 0:20:17 5 4 GB 2.0 Duo 7200 Vista Business Patrick 
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../obj-@CONFIG_GUESS@
ac_add_options --enable-application=browser
mk_add_options MOZ_CO_PROJECT=browser
mk_add_options MOZ_MAKE_FLAGS=-j5
0:59:00 0:20:45 4 4 GB 2.0 Duo 7200 Vista Business Patrick 
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../obj-@CONFIG_GUESS@
ac_add_options --enable-application=browser
mk_add_options MOZ_CO_PROJECT=browser
mk_add_options MOZ_MAKE_FLAGS=-j4
1:15:25 0:21:22 N/A 4 GB 2.0 Duo 7200 Vista Business Patrick 
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../obj-@CONFIG_GUESS@
ac_add_options --enable-application=browser
mk_add_options MOZ_CO_PROJECT=browser
1:03:00 N/A 1 GB 1.6 Duo XP Media Tony
0:55:00 4 2.5 GB 1.6 AMD TurionX2 XP Professional Zaid 
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../obj-@CONFIG_GUESS@
mk_add_options MOZ_MAKE_FLAGS=-j4
ac_add_options --enable-application=browser
ac_add_options --enable-libxul
ac_add_options --disable-tests
ac_add_options --disable-mochitest
ac_add_options --disable-vista-sdk-requirements
real 0:44:29
user 0:13:16
sys 0:07:52 		real 0:07:48
user 0:05:42
sys 0:03:29 		5 2.5 GB 1.6 AMD TurionX2 XP Professional Zaid 
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../obj-@CONFIG_GUESS@
mk_add_options MOZ_MAKE_FLAGS=-j5
ac_add_options --enable-application=browser
ac_add_options --enable-libxul
ac_add_options --disable-tests
ac_add_options --disable-mochitest
ac_add_options --disable-vista-sdk-requirements
real 0:66:34
user 0:13:15
sys 0:07:54 		real 0:09:00
user 0:05:46
sys 0:03:41 		N/A 2.5 GB 1.6 AMD TurionX2 XP Professional Zaid 
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../obj-@CONFIG_GUESS@
ac_add_options --enable-application=browser
ac_add_options --enable-libxul
ac_add_options --disable-tests
ac_add_options --disable-mochitest
ac_add_options --disable-vista-sdk-requirements
1:17:44 - 2 1.5 GB 1.8 AMD Mobile Sempron 5400 Vista Business (32-bit) Nino
0:24:15 0:01:07 N/A 2 GB 2.66 Duo 7200 Ubuntu 8.04 (64-bit) Jason
0:15:12 0:01:04 4 2 GB 2.66 Duo 7200 Ubuntu 8.04 (64-bit) Jason
0:28:18 0:01:04 5 1.5 GB 1.66 Duo 5400 Ubuntu 8.04 (32-bit) Jason 
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../obj-@CONFIG_GUESS@
ac_add_options --enable-application=browser
mk_add_options MOZ_CO_PROJECT=browser
ac_add_options --enable-optimize
ac_add_options --enable-default-toolkit=cairo-gtk2
ac_add_options --disable-tests 
ac_add_options --disable-mochitest
mk_add_options AUTOCONF=autoconf2.13
0:43:14 - - 2 GB 1.80 AMD Mobile Sempron 5400 Ubuntu 8.04 (32-bit) Aaron T 
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../obj-@CONFIG_GUESS@
ac_add_options --disable-optimize
ac_add_options --disable-static --enable-shared
real 0:29:13
user 0:11:26
sys 0:03:47 		real 0:05:8
user 0:04:47
sys 0:01:43 		5 1 GB 2.0 Duo ( OC'd to 2.41 ) 7200 Windows XP Professional ( 32-bit) Chris Bishop 
. $topsrcdir/browser/config/mozconfig
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/ff-opt
mk_add_options MOZ_MAKE_FLAGS=-j5
ac_add_options --disable-tests
real 0:36:50
user 0:10:44
sys 0:03:30 		real 0:05:7
user 0:04:17
sys 0:01:31 		5 1 GB 2.0 Duo ( OC'd to 2.41 ) 7200 Windows XP Professional ( 32-bit) - VS 2008 - Vista SDK Chris Bishop 
. $topsrcdir/browser/config/mozconfig
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/ff-opt
ac_add_options --disable-vista-sdk-requirements
ac_add_options --disable-tests
ac_add_options --disable-accessibility
real 0:27:50
user 0:10:51
sys 0:03:42 		real 0:04:46
user 0:04:21
sys 0:01:32 		5 1 GB 2.0 Duo ( OC'd to 2.41 ) 7200 Windows XP Professional ( 32-bit) - VS 2008 - Vista SDK Chris Bishop 
. $topsrcdir/browser/config/mozconfig
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/ff-opt
mk_add_options MOZ_MAKE_FLAGS=-j5
ac_add_options --disable-vista-sdk-requirements
ac_add_options --disable-tests
ac_add_options --disable-accessibility
Didn't note the time. 
FireFox:
real    16m12.402s
user    2m23.381s
sys     5m11.815s 

ThunderBird
real    16m26.193s
user    2m15.639s
sys     5m4.316s
5 2 GB AMD turion 64x2 Windows Vista Home ( 32-bit) Chinmay Patel 
. 
ac_add_options --enable-application=mail
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/tbird-debug
mk_add_options MOZ_CO_PROJECT=mail,calendar
ac_add_options --disable-optimize
ac_add_options --enable-debug
mk_add_options MOZ_MAKE_FLAGS=-j5
real 33m57.475s
user 31m40.558s
sys 8m36.924s 		real 2m51.103s
user 0m49.142s
sys 1m3.305s 		4 3 GB 667 MHz Intel Core 2 Duo 2.16 GHz 7200 Mac OSX 10.4.11 Ezadkiel V. Marbella 
. $topsrcdir/browser/config/mozconfig
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-ff
ac_add_options --enable-tests
ac_add_options --disable-libxul
ac_add_options --enable-debug --disable-optimize
ac_add_options --enable-shared --disable-static
ac_add_options --with-macos-sdk=/Developer/SDKs/MacOSX10.4u.sdk
mk_add_options MOZ_MAKE_FLAGS="-s -j4"
53:43 22:18 None 1 GB Intel 3.0 GHZ Single Core 7200 Microsoft Windows XP Service Pack 3 Scott Lunel 
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-firefox
mk_add_options MOZ_CO_PROJECT=browser 
ac_add_options --enable-application=browser
ac_add_options --disable-optimize
ac_add_options --enable-debug
ac_add_options --disable-tests
ac_add_options --disable-vista-sdk-requirements

Project News / Updates

09/18/2008 -

  • Use time command to time build time
    • ie. time make -f client.mk build
  • Start collecting data

09/17/2008 -

  • Built the first time in 1h 20m
  • Rebuilt second time in 35m with all apps off
  • Talked with Shaver and directed me to thread on shrinking build time
    • Thread talked about /MP command for building in VS -> not useful in my case
  • Dave's suggestion
    • Understand what each part of the build is taking
    • Collect win32 build times from colleagues

09/16/2008 -

  • Talked with Ted Mielczarek, briefly discussed about where to start and how to approach this project.
    • Start by timing how long each make file is
    • Identify which make file takes a long time

09/15/2008 -

  • Created Project page for this project.
Retrieved from "https://wiki.cdot.senecapolytechnic.ca/w/index.php?title=Profile_the_build_system&oldid=17866"