Difference between revisions of "Firefox Performance Testing : A Python framework for Windows Documentation"
(→Framework Prerequisites) |
(→Framework Configuration) |
||
Line 19: | Line 19: | ||
=Framework Configuration= | =Framework Configuration= | ||
− | #Make sure the prerequisites | + | #Make sure the [[Firefox_Performance_Testing_:_A_Python_framework_for_Windows_Documentation#Framework_Prerequisites | prerequisites]] are installed. |
− | #Download entire framework (directories and all subdirectories) | + | #Make a directory for the framework: |
− | # | + | <pre>$mkdir C:\mozilla</pre> |
+ | #Download entire framework (directories and all subdirectories) and put it in '''C:\mozilla''' | ||
+ | #To setup the appropriate paths for the framework, open the '''paths.py''' file and check if the paths are correct: | ||
*CYGWIN | *CYGWIN | ||
**Path to cygwin bash to get output from dump() in Firefox | **Path to cygwin bash to get output from dump() in Firefox | ||
+ | **By default, the Cygwin you just installed is in your C:\ drive | ||
+ | |||
*SYNC | *SYNC | ||
**Run sync between Ts runs | **Run sync between Ts runs | ||
+ | **By default, the Cygwin you just installed is in your C:\ drive | ||
+ | |||
*BASE_PROFILE_DIR | *BASE_PROFILE_DIR | ||
− | **Path to the base profile directory used for testing. | + | **Path to the base profile directory used for testing. |
+ | **Two ways to configure this: | ||
+ | # Create C:\extension_perf_testing\base_profile, copy all contents of '''base_profile''' folder in '''C:\project\testing\performance\win32\base_profile''' into C:\extension_perf_testing\base_profile | ||
+ | # Configure BASE_PROFILE_DIR to reflect the base_profile path (in this case C:\project\testing\performance\win32\base_profile) | ||
**For the page load test to work, modify the '''hostperm.1''' file, set to allow urls with scheme:file to open in new windows, and the preference to open new windows in a tab should be off. | **For the page load test to work, modify the '''hostperm.1''' file, set to allow urls with scheme:file to open in new windows, and the preference to open new windows in a tab should be off. | ||
+ | |||
*REPORTS_DIR | *REPORTS_DIR | ||
**Path to the directory that the generated reports will go into | **Path to the directory that the generated reports will go into | ||
+ | **Create C:\extension_perf_reports or configure REPORT_DIR path to point to a directory where you want the reports to go into | ||
+ | |||
*INIT_URL | *INIT_URL | ||
**Path to the file url to load when initializing a new profile | **Path to the file url to load when initializing a new profile | ||
+ | |||
*TS_URL | *TS_URL | ||
**The path to the file url to load for startup test (Ts) | **The path to the file url to load for startup test (Ts) |
Revision as of 22:44, 27 November 2006
Framework Prerequisites
You'll need to download some software onto your Windows machine before running these performance tests:
- Python 2.4
- The scripts all run from Python 2.4. You will need the windows version (not the cygwin version).
- Python Win32 Extensions
- These extensions provide some support for process management and performance monitoring.
- Numerical Python
- This is required by Matplotlib.
- Matplotlib
- This library is used to plot the graphs of performance characteristics of Firefox during the page load test.
- PySyc
- This is used for reading/writing config files in YAML.
- Cygwin
- For some reason, we can't get output from Firefox unless it's wrapped in a Cygwin sh shell. So we use cygwin for that, and to sync between Firefox runs.
- msvcp71.dll
- Microsoft® C Runtime Library, v. 7.10.3077.0
- You need a copy of this file to run Python with all the libraries.
- If you have Visual Studio 2003 installed, you've already got it.
Framework Configuration
- Make sure the prerequisites are installed.
- Make a directory for the framework:
$mkdir C:\mozilla
- Download entire framework (directories and all subdirectories) and put it in C:\mozilla
- To setup the appropriate paths for the framework, open the paths.py file and check if the paths are correct:
- CYGWIN
- Path to cygwin bash to get output from dump() in Firefox
- By default, the Cygwin you just installed is in your C:\ drive
- SYNC
- Run sync between Ts runs
- By default, the Cygwin you just installed is in your C:\ drive
- BASE_PROFILE_DIR
- Path to the base profile directory used for testing.
- Two ways to configure this:
- Create C:\extension_perf_testing\base_profile, copy all contents of base_profile folder in C:\project\testing\performance\win32\base_profile into C:\extension_perf_testing\base_profile
- Configure BASE_PROFILE_DIR to reflect the base_profile path (in this case C:\project\testing\performance\win32\base_profile)
- For the page load test to work, modify the hostperm.1 file, set to allow urls with scheme:file to open in new windows, and the preference to open new windows in a tab should be off.
- REPORTS_DIR
- Path to the directory that the generated reports will go into
- Create C:\extension_perf_reports or configure REPORT_DIR path to point to a directory where you want the reports to go into
- INIT_URL
- Path to the file url to load when initializing a new profile
- TS_URL
- The path to the file url to load for startup test (Ts)
- TP_URL
- The path to the file url to load for page load test (Tp)
- Create a YAML config file (config.yaml) with info about the profiles you want to test. Copy and paste the following into config.yaml:
# Filename will be appended to the timestamp in the report filename. # Use letters and underscores only filename: slowmachine # The title of the report title: 1 GHz Celeron # Name of profile to test Test profile 1: # Path to Firefox to test firefox: C:\Program Files\Mozilla Firefox 2 Beta 1\firefox.exe # Preferences to set in the test (use "preferences : {}" for no prefs) preferences: javascript.options.showInConsole : true xpinstall.enabled : true # Extensions to install in test (use "extensions: {}" for none) extensions: # Need quotes around guid because of curly braces "{12345678-1234-1234-1234-abcd12345678}" : c:\path\to\unzipped\xpi foo@sample.com : c:\path\to\other\unzipped\xpi
- Note:
- Check that your firefox path is right
- Preferences and extensions can be left as none:
- use "preferences : {}" for no prefs
- use "extensions: {}" for none
5. Run run_tests.py with the name of your config file as an argument. You can use a space-separated list of config files, to generate a report of startup and page load times. The report will be placed in the REPORTS_DIR directory you specified in paths.py.
DIRECTORY STRUCTURE:
page_load_test/ This directory contains the JavaScript files and html data files for the page load test. The page load test opens a new window and cycles through loading each html file, timing each load. startup_test/ This directory contains the JavaScript to run the startup test. It measures how long it takes Firefox to start up. base_profile/ This directory contains the base profile used for testing. A copy of this profile is made for each testing profile, and extensions or prefs are added according to the test_configs array in run_tests.py. For the page load test to run correctly, the hostperm.1 file must be set to allow scheme:file uris to open in new windows, and the pref to force a window to open in a tab must not be set. The dom.allow_scripts_to_close_windows pref should also be set to true. The browser.shell.checkDefaultBrowser pref should be set to false. run_tests.py, paths.py These files should be configured to run the test on different machines, with different extensions or preferences. See setup above.