Project Name
PGO Related Bugs
Project Description
Firefox crashes or experiences bugs after creating PGO builds on Windows. The focus of this project is to determine what is causing these crashes or bugs and either patch/fix them or find solutions to allow for the optimizations without incurring the problems.
As of Oct 2nd:
Profiling Firefox to increase the number of functions optimized in the PGO build of Firefox.
Project Leader(s)
Project Contributor(s)
Contributors are needed for help on profiling Firefox to improve PGO on Firefox.
What contributions are needed?
- What should be profiled?
- A write up on steps to produce profile.
- Creating scripts to execute profile steps.
| Name
|
|
Contribution
|
| Chris Bishop ( Example )
|
|
Script to open and close Firefox.
|
Project Tasks
| Task |
Details |
Priority |
Status |
Target |
Link(s) or Notes |
| Compile Firefox PGO |
Compile the trunk of Firefox using make -f client.mk profiledbuild. |
High |
Done |
0.1 R |
|
| Reproduce PGO Bugs or Crashes |
I need to reproduce the bugs or crashes other people were experiencing with their pgo builds. |
High |
Done |
0.1 R |
Unable to reproduce. Focusing on profiling Firefox. Blog Post |
| Compile Firefox PGO Tests |
Compile Firefox with VS 2005 and Vista SDK and compile with VS 2008 w/o Vista SDK. |
Low |
Not Started |
0.3 R |
|
| Determine 0.1 Release |
I need to determine what I'll be releasing for the 0.1 Release |
High |
On Going |
Sept 28th, 2008 |
|
Project Details
Testing Scripts
|
|
| The scripts I am using for running tests against my builds ( Oct 2nd, 2008 )
|
|
#!/bin/sh
####### the .mozconfig file to use for this build
export MOZCONFIG=/f/mozilla/.mozconfigPGO
####### the subdirectory trunk holds my src files
cd trunk
####### time the profile build and output stdout and stderr to files that are timestamped
time make -f client.mk profiledbuild 1> /i/mozilla/pgo-output-`date +%s`.txt 2> /i/mozilla/pgo-errors-`date +%s`.txt
####### after the build is completed done run the xpcshell tests and output stdout and stderr to file that are timestamped
make check 1> /i/mozilla/pgo-check-output-`date +%s`.txt 2> /i/mozilla/pgo-check-errors-`date +%s`.txt
####### change to the objdir directory then to the mochitest directory
cd /i/mozilla/ff-pgo/_tests/testing/mochitest
####### run the mochitest, run test right away, close the browser when done, log all output to a timestamped file
python runtests.py --autorun --close-when-done --log-file=/i/mozilla/pgo-mochitest-`date +%s`.log --file-level=DEBUG
|
| Here is the .mozconfig file ( Oct 2nd, 2008 )
|
|
. $topsrcdir/browser/config/mozconfig
mk_add_options MOZ_OBJDIR=/i/mozilla/ff-pgo
mk_add_options MOZ_MAKE_FLAGS=-j5
####### this is the script to run after the first build to gather profile data
mk_add_options PROFILE_GEN_SCRIPT="sh /f/mozilla/pgo.sh"
ac_add_options --disable-vista-sdk-requirements
####### this has to be enabled to perform the mochitests
ac_add_options --enable-tests
ac_add_options --disable-debug
ac_add_options --disable-accessibility
|
| Here is the script the profile build runs to gather profile data ( Oct 2nd, 2008 )
|
#!/bin/sh
export NO_EM_RESTART=1
mkdir $OBJDIR/_profileprofile
cd /i/mozilla/ff-pgo/_tests/testing/mochitest
python runtests.py --autorun --close-when-done --log-file=/i/mozilla/pgo-profile-mochitest-`date +%s`.log --file-level=DEBUG
|
|
Related Bugs
|
|
Figure out why building jemalloc breaks with PGO enabled - 419470
|
|
|
sort out PGO-triggered bugs - 419893
|
|
|
win32 mozilla-central weirdness with PGO - 437002
|
Releases
|
|
0.1 Release
|
|
- Reproduce bug(s) - Update: Unable to.
Document how to reproduce bug(s).Hone in on cause of bug(s).- Running xpcshell and Mochitest against PGO build.
- Create strategy for profiling Firefox to improve performance.
More to come....
|
0.2 Release
0.3 Release
|
Errors
|
|
- Build Environment:
- Windows XP (32-bit)
- Visual Studio 2008
- Vista SDK
| Firefox Compilation Errors ( Sept 26th, 2008 )
|
e:\mozilla\trunk\modules\lcms\src\cmscgats.c(875) : fatal error C1001: An internal error has occurred in the compiler.
(compiler file 'f:\dd\vctools\compiler\utc\src\p2\main.c[0x511953E5:0x00000008]', line 182)
To work around this problem, try simplifying or changing the program near the locations listed above.
Please choose the Technical Support command on the Visual C++
Help menu, or open the Technical Support help file for more information
LINK : fatal error LNK1000: Internal error during IMAGE::BuildImage
Version 9.00.21022.08
ExceptionCode = C0000005
ExceptionFlags = 00000000
ExceptionAddress = 511953E5 (51030000) "e:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\c2.dll"
NumberParameters = 00000002
ExceptionInformation[ 0] = 00000000
ExceptionInformation[ 1] = 00000008
CONTEXT:
Eax = 00000000 Esp = 0012ED18
Ebx = 02D5D215 Ebp = 0012ED2C
Ecx = 02D96601 Esi = 02D5D280
Edx = 02DAAE12 Edi = 00000305
Eip = 511953E5 EFlags = 00010246
SegCs = 0000001B SegDs = 00000023
SegSs = 00000023 SegEs = 00000023
SegFs = 0000003B SegGs = 00000000
Dr0 = 00000000 Dr3 = 00000000
Dr1 = 00000000 Dr6 = 00000000
Dr2 = 00000000 Dr7 = 00000000
|
| Firefox Compilation Errors ( Sept 22th, 2008 )
|
make[6]: *** [mozlcms.dll] Error 232
make[6]: *** Deleting file `mozlcms.dll'
make[5]: *** [libs] Error 2
make[4]: *** [libs_tier_external] Error 2
make[3]: *** [tier_external] Error 2
make[2]: *** [default] Error 2
make[1]: *** [build] Error 2
make: *** [profiledbuild] Error 2
|
|
Performance
|
|
| PGO Build Results ( Sept 20th, 2008 )
|
The program loads 15,000 random records from a text file into 4 tables each using a different method for storage and searching.
- Simple Table: Uses an array and a linear search.
- Chain Table: Uses an array of link lists. Search is done with a Hash Key.
- Hash Table: Uses an array and searches with linear probing. Also uses a Hash Key.
- Tree Table: Uses a binary search tree.
After the data is loaded into a table, 30 different tests are run against it. The same 30 tests for each table.
Here are the Results.
|
|
Non-PGO
|
PGO
|
Improved By
|
| Simple Table:
|
28 Secs |
4 Secs
|
7x
|
| Chain Table:
|
0.109 Secs |
0.047 Secs
|
2.3x
|
| Hash Table:
|
0.141 Secs |
0.047 Secs
|
2.3x
|
| Tree Table:
|
0.287 Secs |
0.078 Secs
|
3.6x
|
|
|
Project News
October 2008
| Date
|
|
Comments
|
| Oct 4nd, 2008
|
|
A build of Firefox without PGO resulted in 35 failed Tests in Mochitest. Xpcshell passed just fine. At this time the tree is closed so I cannot get the latest bug free version.
|
| Oct 2nd, 2008
|
|
After talking with Ted and Dave Humphrey I am going to focus my project on increasing the profiling of Firefox to increase its speed.
I am also enabling the modules that have PGO disable and running the xpcshell and Mochitests on them to determine if they are still broken or if they are now working.
|
September 2008
| Date
|
|
Comments
|
| Sept 30th, 2008
|
|
Failed to reproduce bugs. I followed Ted's suggestions on my blog but still not experiencing any problems with my pgo build.
|
| Sept 28th, 2008
|
|
Ted pointed me in the right direction on how to reproduce the PGO bugs. His post is here.
|
| Sept 26th, 2008
|
|
Successfully compiled Firefox with PGO using Visual Studio 2005 compiler. Tests and debug are disabled.
|
| Sept 26th, 2008
|
|
On IRC some guys suggested compiling with Visual Studio 2005 instead.
|
| Sept 24th, 2008
|
|
Attempted to compile with debug and test disabled. Still Failed.
|
| Sept 22th, 2008
|
|
Failed to compile Firefox with PGO on Visual Studio 2008.
|
| Sept 20th, 2008
|
|
Successfully compiled one old assignment using PGO in Visual Studio. Results in Project Details.
|
| Sept 19th, 2008
|
|
Tried to compile some old assignments using PGO in Visual Studio. Two assignments failed.
|
| Sept 18th, 2008
|
|
Created this Project Page.
|
| Sept 17th, 2008
|
|
Created First Build of Firefox.
|
| Sept 15th, 2008
|
|
Met with Ted Mielczarek and discuss PGO Project.
|
