New Firefox Performance Testing Documentation

Introduction

This documentation is still in progress.

• This document assumes that you have the following:
  • msvcp71.dll (NOTE: If you have Visual Studio 2003 installed, you've already got it.)
    • Microsoft® C Runtime Library, v. 7.10.3077.0
    • You need a copy of this file to run Python with all the libraries.
• Put it in your C: directory

• This documentation is the revised version of Annie Sullivan's Readme.txt for the Firefox Performance Testing.

• Unfortunately, the Firefox Performance Testing is only catered to Windows users at this time.

Quick start

3 easy steps to run Firefox Performance Testing:

• Install Cygwin
• Save the script
• Run the script

Cygwin Setup

• Cygwin
  • Download cygwin.
  • Install cygwin with the DOS-style line endings.
  • At minimum, you are required to have the following additional tools:
    • unzip (Archive category)
    • wget (Web category)

unzip (Archive category)


wget (Web category)

Note: For some reason, we can't get output from Firefox unless it's wrapped in a Cygwin sh shell. So we use cygwin for that, and to sync between Firefox runs.

Configuration

Save the script

• Save this script at a desired location
  • Script: Script to configure Firefox Performance Testing Framework
• Edit the script to your own choosing. You may provide a list of extensions to install and test in the config.yaml section.
• Note: This script will run the performance testing. If don't wish for that to happen, simply comment out the last line of the script (to comment out, use #):

#./run_tests.py config.yaml;

• Refer to Run Firefox Performance Tests if you wish to run the performance tests later.

Run the script

• In cygwin, browse to the location where you saved your script.
• Before you run the script, refer to What happens when you run Firefox Performance Tests
• From cygwin, run the script.

For example:

$./perfconfig.sh

What the script does

• Creates directories needed for the performance testing
  • Directory that holds the files for the performance testing
  • Reports directory to hold the results generated
• Downloads and installs pre-requisites
• Write to config.yaml

Config File (config.yaml)

Config File Info

Your config.yaml should consist of these items.

• filename
• title
• (Test Profile name) - any name
  • firefox
  • preferences
  • extensions

Refer to sample.config:

     # Filename will be appended to the timestamp in the report filename.
     # Use letters and underscores only
     filename: yourfilename
     # The title of the report
     title: 1 GHz Celeron

     # Name of profile to test
     Test profile 1:
       # Path to Firefox to test
       firefox: C:\Program Files\Mozilla Firefox 2 Beta 1\firefox.exe

       # Preferences to set in the test (use "preferences: {}" for no prefs)
       preferences:
         javascript.options.showInConsole : true
         xpinstall.enabled : true

       # Extensions to install in test (use "extensions: {}" for none)
       extensions:
         # Make sure that the path is directed to the desired location of the extension
         # Need quotes around guid because of curly braces
         "{12345678-1234-1234-1234-abcd12345678}" : c:\path\to\unzipped\xpi
         foo@sample.com : c:\path\to\other\unzipped\xpi

Config File Instructions

1. Verify that the firefox path is pointing to the right Firefox instance on your system.

• You will hit a zero division error and some other errors if you don't point it to the right path

2. preferences:

• If you don't have preferences, change to:

  preferences : {}

  (make sure there is a space between ":" and "{" or else it will cause a syntax error)
• If you do have preferences, refer to sample.config

3. extensions:

• If you don't have extensions to install, change to:

  extensions: {}

  (make sure there is a space between ":" and "{" or else it will cause a syntax error)
• If you do have extensions to install, refer to this sample:

       extensions:
         # Make sure that the path is directed to the desired location of the extension
         # Need quotes around guid because of curly braces
         "{12345678-1234-1234-1234-abcd12345678}" : c:\path\to\unzipped\xpi
         foo@sample.com : c:\path\to\other\unzipped\xpi 

• Make sure that the path to the extension is right

4. Change filename and title to whatever you desire

Run Firefox Performance Tests

• By default, the run_tests.py file that you need to execute to run the Firefox Performance Testing is located at c:\mozilla\testing\performance\win32\
• You can run run_tests.py either from the DOS prompt or Cygwin:

DOS

$run_tests.py config.yaml

Cygwin

./run_tests.py config.yaml

What happens when you run Firefox Performance Tests

• All opened browser windows will be closed automatically!
• You might run into this message on your browser: This page should close Firefox. If it does not, please make sure that the dom.allow_scripts_to_close_windows preference is set to true in about:config
  • Type auto:config in the address bar and double click on dom.allow_scripts_to_close_windows to change it to true
• If you see processing... Performance Testing is starting, performance testing is running..
• You can check the progress by the progress bar