Mozilla Metrics Server Collection

Project Name

Mozilla Metrics Server Collection

Project Description

This project basically proposes to recieve the data being collected by the Metrics extension and store it within a database. In the future, possibly use the data and display it in some form.

Project Leader(s)

Simon Jung (simonJ)

Project Contributor(s)

Currently none; however in the future I might need testers. Please contact me if you would like to contribute.

Project Details

PROJECT SPECIFICATIONS FOR VERSION 0.1

This project will require me to do the following:

• Download and patch metrics extension
• Compile the metrics extension with Minefield
• Intialize the metrics extension using a config file
• Send a config file from server to client extension
• Recieve data from the metrics extension
• Save recieved data to a file

OVERVIEW

On initialization of the metrics extension, the extension pings the server for a config file; which is in the form of an xml. This config file is sent back to the extension as a server response and includes a http header along with the config file. The server response will tell the extension two things - a STATUS 200 to say everything is OK, a Content-Type: application/xml to inform that the data being sent is an xml file. Once the extension recieves an OK, the extension will then take the config file and save it into the users Firefox profile folder. If there is a failure, STATUS 3xx, 4xx, or 5xx, the extension will create a config file that disables the extension. Once all these requirements are met, the extension will start collecting data and can be seen by checking "about:metrics".

If there was any data collected when the timer reaches the upload interval (described in the xml file), the extension will BZIP the xml file and HTTP POST the data to the server with a "Content-Type: application/vnd.mozilla.metrics.bz2" to indicate the type of data being sent. The extension will then wait for a new config file to reset the extension. The reset basically clears the old data and resets the config file. This process will continue in a loop.

SAMPLE DATA

Config file sample

Data collected sample

Server CGI Script

Metrics Patch

Project COMPLICATIONS & SOLUTIONS

• There seems to be a link problem when trying to compile the extension in windows xp. Reading the source code, and with the help of luser, I was able to figure out some problems within the code. Minor changes were made to make files in both the "/extension/metrics/build" and "/extensions/metrics/test" folders; Added "USE_STATIC_LIBS=1" so that the compiler knows to use the static library. Will create a patch and post on bugzilla once I'm done.

• Echoing a file back to the extension was pretty straightforward; however, recieving the proper data back was more difficult. Initially, I assumed the data being sent back was an xml file but after reading the source code in-depth, I found that the data was actually compressed with BZIP2 (x.bz2). Changes were made to the CGI to bunzip2 the data.

• Another problem, similar to problem 2, was that my while loop to read stdin was not recieving all the data properly. With the help of "ctyler" I was able to recieve the data in-tact.

Project News

Thursday, October 18th 2007

• Finally compiled a working version of Minefield; however there is a problem compiling the Metrics extension; error LNK2005.

Friday, October 19th 2007

• Fixed the link errors with minor changes to the source code (detail above)
• Attempted to create a server script that can handle the request from the extension.

Saturday, October 20th 2007

• Server script is initializing the extension; however all attempts to read the data being sent to the server has failed. The data seems to be jibberish.

Saturday, October 20th 2007

• Created a simple html page to test the server script. The html page basically allows me to send xml docs through a http post.
• Tested server script to see if it is getting the right data.

Sunday, October 21st 2007

• The server script and extension are finally working together fine.
• A config file is being sent back and the data is being recieved.
• Trying to build the extension with the new patch in Windows XP

Monday, October 22nd 2007

• Finally got the metrics extension built and working with the nightly.
• Created a patch for the metrics extension.
• Going to submit a patch to bugzilla after I get somone to compile it in linux

Resources

Project specifications by Alex Polvi:

• RequirementsV01

Bugs related to project at Bugzilla:

• BUG399869

• BUG393881

• BUG339528

• BUG332947

Source Code:

• Metrics Source Code @ MXR Mozilla