Firefox Performance Testing Lab Fall 2010
Contents
Goal
The goal of this lab is twofold. First, to provide students with a real-world experience of working collaboratively in an open community; and second, to work on a cutting-edge, but manageable project within the Mozilla community.
Objective
To conduct A/B performance tests of the Chrome Experiments using nightly builds of both Firefox and Chrome, in order to identify performance bottlenecks in Firefox. Also, to profile and file bugs in order to fix these issues.
Method
- As a group determine a method for dividing the Chrome Experiments so they all get tested
- Install both the Firefox and Chrome nightly builds
- Test each experiment in the two browsers, looking for various issues:
- Speed - is Firefox as fast as Chrome at rendering the graphics?
- Smoothness - are the graphics as smooth as in Chrome? Do you notice a lot of pauses, jerkiness, etc.?
- Responsiveness - does Firefox remain responsive while you run the demo? Is it pegging your CPU(s)?
- Record your findings, as well as browser and computer info ([about:support], [about:]) in the Results Spreadsheet.
Resources
- Chrome Experiments
- Results Spreadsheet. Talk to Dave to get permissions in order to edit.
- Firefox (i.e., Minefield) nightly builds
- Chrome (i.e., Chromium) nightly builds
- Example performance bug
Tests: Initial First Round
(humph) Let's refine this a bit more, such that we track:
Name of Experiment | URL of Experiment | Your Name | Date of Test | Hardware Info | Browser Info | Firefox Performance - Speed | Firefox Performance - Smoothness | Firefox Performance - Responsiveness | Notes and other Details |
(carl) Separate or cumulative results page?
| Test No. | Name |
Results |
|---|---|---|
| 1-10 | Carl | Results |
| 11-20 | Kevin Lasconia | Results |
| 21-30 | Andrew Condinho | Results |
| 31-40 | Stephen Bologna | Results |
| 41-50 | Kenneth Pangilinan | Results |
| 51-60 | Pete Leaning | Results |
| 61-70 | Brian Law | Results |
| 71-80 | Steven Weerdenburg | Results |
| 81-90 | James Evangelista | Results |
| 91-100 | Kaitlyn Callow | Results |
| 101-110 | Crystal de Nobrega | Results |
| 111-120 | Chris DeCairos | Results |
| 1-26 (Linux) | Konstantin Novichikhin | Results |
Tests: Second Round
For each experiment you tested as part of the first-round of testing that was not as fast or faster than Chrome, please create an entry in the table below. Include details about what you are seeing, what is failing, etc. Also, if you need to file a bug, include the link to the bug you filed.
| Test | Tester | Problem | Additional Info |
|---|---|---|---|
| Lorenz 84 | Stephen Bologna | In the 32bit version of Minefield on Vista the browser froze for several seconds when the page loaded, and any attempt to interact with the test caused it to freeze again. In the 64bit version of Minefield on Window 7, the page took several minutes to load, and the image was not drawn properly. | |
| FlowerPower | Kaitlyn Callow |
|
|
| Cheloniidae Live | Kaitlyn Callow | Not running. | Returning error:
Error: console is not defined Source File: http://spencertipping.com/beta/cheloniidae-live-b1/script.js Line: 2 |
| PlasmaTree | Kaitlyn Callow | Considerably faster in Chrome when you click multiple times to draw trees at same time. | |
| Liquid Particles | Steven Weerdenburg | Minefield particle dot redraw very slow, manages approx 10 frames/sec | Chromium has no delay when rendering particle dot movement. Both have difficulty in letter rendering |
| BallDroppings | Carl D | far more audio clipping and visual chop when at a 'max' ball drop speed when compared to chrome | |
| VP Puzzle | Carl D | far greater delay generating picture puzzle windows | Firefox was much faster and smoother then Chrome when creating the one video puzzle and its windows |
| Ball Pool | Carl D | Choppy at 50+% ball saturation when moving the window | Chromes breakpoint was far later |
| ChromeDrones | Carl D | Video and Audio chop when 8+ notes were introduced | |
| Liquid Particles | Steven Weerdenburg | Minefield particle dot redraw very slow, manages approx 10 frames/sec | Chromium has no delay when rendering particle dot movement. Both have difficulty in letter rendering |
| Browser Pong | Steven Weerdenburg | Minefield calculates the "ball" window height as twice that of Chromium | |
| Realtime Video->ASCII Conversion | Steven Weerdenburg | Seems to have very difficult time at higher "resolutions" (smaller fonts). Scales up in canvas size ok, but becomes unresponsive over prolonged periods (5+ minutes) of use. | |
| Colorscube | Kevin Lasconia | When the cube is rotated in any direction the animation is very choppy. There is also a delay between moving the mouse in one direction and the actual cube moving in that direction. | Chrome was very smooth, and responsive. |
| Monster | Kevin Lasconia | In Firefox, when more complex objects are generated the spinning animation of the object becomes increasingly more choppy. During transformations the animation would freeze for a few seconds then continue. | Chrome did not experience any of the outlined problems above. |
| Browser Ball | Kevin Lasconia | There is an issue when multiple windows are spawned in Firefox for this experiment. When moving the ball from the main window to another the ball will get stuck. Even when trying to "throw" the ball to another window it will still get stuck. | In Chrome, the ball can actually be moved to and from new windows. |
| Gravity | Kevin Lasconia | This experiment does not work in the Mozilla/5.0 (Windows NT 6.0; WOW64; rv:2.0b7pre) Gecko/20100916 Firefox/4.0b7pre nightly build. | It works fine in Chrome. It also works in Firefox 3.6.10. |
Bug Reports
NOTE: All bugs related to what we find should have [chromeexperiments] in the bug's Whiteboard field, so we can track them. The following Bugzilla search will list them all: https://bugzilla.mozilla.org/buglist.cgi?quicksearch=[chromeexperiments]
