1
edit
Changes
→Introduction
[[Firefox_Performance_Testing_:_A_Python_framework_for_Windows | Back to Firefox Performance Testing Project]]
=Introduction=
This documentation is the revised version of Annie Sullivan's [[Readme.txt]] for the Firefox Performance Testing.
Unfortunately, the Firefox Performance Testing is only catered to Windows users at this time.
=Hardware Requirements=
*Win32 system
=Framework Prerequisites=
(List the sequence and provide sh script to automate this)
You'll need to download these before running performance testing:
* [http://www.python.org/ftp/python/2.4/python-2.4.msi Python 2.4]
* [http://www.cygwin.com/setup.exe Cygwin]
**For some reason, we can't get output from Firefox unless it's wrapped in a Cygwin sh shell. So we use cygwin for that, and to sync between Firefox runs.
* [http://www.dll-filesdriverskit.com/dllindexdll/downloadmsvcp71.php?msvcp71download0UIgUIXDmS dll/2371.html msvcp71.dll](NOTE: If you have Visual Studio 2003 installed, you've already got it.)
**Microsoft® C Runtime Library, v. 7.10.3077.0
**You need a copy of this file to run Python with all the libraries. **If you have Visual Studio 2003 installed, you've already got it.
=Framework Configuration=
==Configure the environment==
#Make sure the [[Firefox_Performance_Testing_:_A_Python_framework_for_Windows_Documentation#Framework_Prerequisites | prerequisites]] are installed.(sh script will automate this part - but get user to check)
#Make a directory for the framework:<pre>$mkdir C:\mozilla</pre>
#Download entire framework [http://www.yorku.ca/abel/liz/mozilla/Win32PerfTesting_v4.zip Performance Testing Framework] (directories and all subdirectories) and put it in '''C:\mozilla'''(one of two options 1.put reports dir in the framework 2.check if it exists and prompt user if he/she wants it created)
==Configure the paths (paths.py)==
===CYGWIN===
*Path Verify that the path is pointing to the desired cygwin directory<pre>CYGWIN = r'c:\cygwin\bin\bash.exe -c'</pre>*The path to cygwin bash to get output from dump() in Firefox*By default, If you didn't specify a different root directory for the Cygwin you just to be installed , it is in your '''C:\ drive'''
===SYNC===
*Run Verify that the path is pointing to the desired cygwin directory<pre>SYNC = r'c:\cygwin\bin\sync'</pre>*Runs sync between Ts runs*By default, If you didn't specify a different root directory for the Cygwin you just to be installed , it is in your '''C:\ drive'''
===BASE_PROFILE_DIR===
*Path to the base profile directory used for testing.*Check Verify that BASE_PROFILE_DIR is pointing to the '''base_profile''' directory*If you didn't alter the dir directory structure, it the '''base_profile''' path would be: ''C:\mozilla\testing\performance\win32\base_profile<pre>BASE_PROFILE_DIR = r'C:\mozilla\testing\performance\win32\base_profile''' </pre>*IMPORTANT: Check that there is content in the '''base_profile ''' directory
*(Optional) For the page load test to work, modify the '''hostperm.1''' file
**Set to allow urls with scheme:file to open in new windows
**Set the preference to open new windows in a tab should be off.
===REPORTS_DIR===*Path This is the path to the directory that the generated reports will go into*<pre>REPORTS_DIR = r'C:\mozilla\testing\performance\win32\extension_perf_reports'</pre>*Create '''extension_perf_reports''' dir in '''C:\mozilla\testing\performance\win32\'''
OR
**Configure Set '''REPORT_DIR ''' path to a directory where you want the reports to go into. Make sure that directory is created.
<pre>INIT_URL = 'file:///c:/mozilla/testing/performance/win32/initialize.html'</pre>
*Check that '''initialize.html''' is in '''C:\mozilla\testing\performance\win32\'''
*'''Note: Don't change the local file url to a file path'''
<pre>TS_URL = 'file:///c:/mozilla/testing/performance/win32/startup_test/startup_test.html?begin='</pre>
*Check that '''startup_test.html''' is in '''C:\mozilla\testing\performance\win32\startup_test'''
*'''Note: Don't change the local file url to a file path'''
<pre>TP_URL = 'file:///c:/mozilla/testing/performance/win32/page_load_test/cycler.html'</pre>
<pre>
# Filename will be appended to the timestamp in the report filename.
# Extensions to install in test (use "extensions: {}" for none)
extensions:
# Make sure that the path is directed to the desired location of the extension
# Need quotes around guid because of curly braces
"{12345678-1234-1234-1234-abcd12345678}" : c:\path\to\unzipped\xpi
foo@sample.com : c:\path\to\other\unzipped\xpi
</pre>
==Instructions==1. Verify that the '''firefox''' path is pointing to the right Firefox instance on your system.* You will hit a zero division error and some other errors if you don't point it to the right path 2. '''preferences''':* If you don't have '''preferences''', change to: <pre>preferences : {}</pre> (make sure there is a space between ":" and "{" or else it will cause a syntax error)* If you do have '''preferences''', refer to [[sample.config]] 3. '''extensions''':* If you don't have '''extensions''' to install, change to: <pre>extensions: {}</pre> (make sure there is a space between ":" and "{" or else it will cause a syntax error)* If you do have '''extensions''' to install, refer to this sample:<pre> extensions: # Make sure that the path is directed to the desired location of the extension # Need quotes around guid because of curly braces "{12345678-1234-1234-1234-abcd12345678}" : c:\path\to\unzipped\xpi foo@sample.com : c:\path\to\other\unzipped\xpi </pre>* Make sure that the path to the extension is right 4. Change '''filename''' and '''title''' to whatever you desire 5. Run In '''cmd''', run '''run_tests.py''' with the name of your config file as an argumentin '''c:\mozilla\testing\performance\win32\''': <pre>$run_tests.py config.yaml</pre> * Running the test will close any existing Firefox windows. (this part has to be highlighted)**You might run into this message on your browser: '''This page should close Firefox. If it does not, please make sure that the dom.allow_scripts_to_close_windows preference is set to true in about:config''' **Type '''auto:config''' in the address bar and double click on '''dom.allow_scripts_to_close_windows''' to change it to true* If you see '''processing... Performance Testing is starting''', performance testing is running.. wait.. * After the performance testing is done, your reports will be placed in the [[Firefox_Performance_Testing_:_A_Python_framework_for_Windows_Documentation#REPORTS_DIR | '''REPORTS_DIR''']] directory you specified in '''paths.py''' *You can use a space-separated list of config files, to generate a report of startup and page load times. The report will be placed in the REPORTS_DIR directory you specified in paths.py.
=Directory Structure of Framework=
==Overview of Structure==
A glance at the Framework File Structure (CVS files not included):
<pre>
win32
|
|
|__ base_profile (dir)
| |
| |__ bookmarkbackups (dir)
| | |
| | |__ .html files
| |
| |__ Cache (dir)
| |
| |__ .bak, .html, .ini, .dat, .txt, .js, .rdf, .mfl files
|
|
|__ page_load_test(dir)
| |
| |__ base(dir)
| | |
| | |__ other dirs and .html files
| |
| |__ cycler.html & report.html
|
|
|__ startup_test
| |
| |__ startup_test.html
|
|
|__ extension_perf_reports (dir for generated reports)
|
|
|__ run_tests.py, paths.py, config.yaml and other .py, .html files
</pre>
*NOTE: Content in '''base_profile''' dir may vary
The following is written by Annie Sullivan (annie.sullivan@gmail.com):
==base_profile/==
*This directory contains the base profile used for testing.
*A copy of this profile is made for each testing profile, and extensions or prefs are added according to the test_configs array in run_tests.py.
*For the page load test to run correctly, the hostperm.1 file must be set to allow scheme:file uris to open in new windows, and the pref to force a window to open in a tab must not be set.
*The dom.allow_scripts_to_close_windows pref should also be set to true. The browser.shell.checkDefaultBrowser pref should be set to false.
==page_load_test/==
*This directory contains the JavaScript files and html data files for the page load test. *The page load test opens a new window and cycles through loading each html file, timing each load.
==startup_test/==
*This directory contains the JavaScript to run the startup test. *It measures how long it takes Firefox to start up. ==extension_perf_report/==*This directory is where the generated report will go into. *You may specify another directory to substitute this directory, but make sure that the '''paths.py''' is changed to point to it.
==run_tests.py, paths.py==
*These files should be configured to run the test on different machines, with different extensions or preferences. See setup above.