New Firefox Performance Testing Documentation

Introduction

This documentation is still in progress.

This document assumes that you have the following:

• msvcp71.dll(NOTE: If you have Visual Studio 2003 installed, you've already got it.)
  • Microsoft® C Runtime Library, v. 7.10.3077.0
  • You need a copy of this file to run Python with all the libraries.

This documentation is the revised version of Annie Sullivan's Readme.txt for the Firefox Performance Testing.

Unfortunately, the Firefox Performance Testing is only catered to Windows users at this time.

Quick start

3 easy steps to run Firefox Performance Testing:

• Install Cygwin
• Save the script
• Run the script

Cygwin Setup

• Cygwin
  • Download cygwin.
  • Install cygwin with the DOS-style line endings.
  • At minimum, you are required to have the following additional tools:
    • unzip (Archive category)
    • wget (Web category)

unzip (Archive category)


wget (Web category)

Note: For some reason, we can't get output from Firefox unless it's wrapped in a Cygwin sh shell. So we use cygwin for that, and to sync between Firefox runs.

Configuration

Save the script

• Save this script at a desired location
  • Script: Script to configure Firefox Performance Testing Framework
• Edit the script to your own choosing. You may provide a list of extensions to install and test in the config.yaml section.
• Note: This script will run the performance testing. If don't wish for that to happen, simply comment out the last line of the script (to comment out, use #):

#./run_tests.py config.yaml;

• Refer to Run Firefox Performance Tests if you wish to run the performance tests later.

Run the script

• In cygwin, browse to the location where you saved your script.
• Before you run the script, refer to What happens when you run Firefox Performance Tests
• From cygwin, run the script.

For example:

$./perfconfig.sh

What the script does

• Creates directories needed for the performance testing
  • Directory that holds the files for the performance testing
  • Reports directory to hold the results generated
• Downloads and installs pre-requisites
• Write to config.yaml

Config File (config.yaml)

Config File Info

Your config.yaml should consist of these items.

• filename
• title
• (Test Profile name) - any name
  • firefox
  • preferences
  • extensions

Refer to sample.config:

     # Filename will be appended to the timestamp in the report filename.
     # Use letters and underscores only
     filename: yourfilename
     # The title of the report
     title: 1 GHz Celeron

     # Name of profile to test
     Test profile 1:
       # Path to Firefox to test
       firefox: C:\Program Files\Mozilla Firefox 2 Beta 1\firefox.exe

       # Preferences to set in the test (use "preferences: {}" for no prefs)
       preferences:
         javascript.options.showInConsole : true
         xpinstall.enabled : true

       # Extensions to install in test (use "extensions: {}" for none)
       extensions:
         # Make sure that the path is directed to the desired location of the extension
         # Need quotes around guid because of curly braces
         "{12345678-1234-1234-1234-abcd12345678}" : c:\path\to\unzipped\xpi
         foo@sample.com : c:\path\to\other\unzipped\xpi

Config File Instructions

1. Verify that the firefox path is pointing to the right Firefox instance on your system.

• You will hit a zero division error and some other errors if you don't point it to the right path

2. preferences:

• If you don't have preferences, change to:

  preferences : {}

  (make sure there is a space between ":" and "{" or else it will cause a syntax error)
• If you do have preferences, refer to sample.config

3. extensions:

• If you don't have extensions to install, change to:

  extensions: {}

  (make sure there is a space between ":" and "{" or else it will cause a syntax error)
• If you do have extensions to install, refer to this sample:

       extensions:
         # Make sure that the path is directed to the desired location of the extension
         # Need quotes around guid because of curly braces
         "{12345678-1234-1234-1234-abcd12345678}" : c:\path\to\unzipped\xpi
         foo@sample.com : c:\path\to\other\unzipped\xpi 

• Make sure that the path to the extension is right

4. Change filename and title to whatever you desire

Run Firefox Performance Tests

• By default, the run_tests.py file that you need to execute to run the Firefox Performance Testing is located at c:\mozilla\testing\performance\win32\
• You can run run_tests.py either from the DOS prompt or Cygwin:

DOS

$run_tests.py config.yaml

Cygwin

./run_tests.py config.yaml

What happens when you run Firefox Performance Tests

• All opened browser windows will be closed automatically!
• You might run into this message on your browser: This page should close Firefox. If it does not, please make sure that the dom.allow_scripts_to_close_windows preference is set to true in about:config
  • Type auto:config in the address bar and double click on dom.allow_scripts_to_close_windows to change it to true
• If you see processing... Performance Testing is starting, performance testing is running..
• You can check the progress by the progress bar