3
edits
Changes
m
*Looked into a batch file for automating configuration, gave pointers
*Tested and commented on the new framework (version 2)
[[User: robcee | Rob Campbell]] (robcee)
*Gave me a number of python tips :)
[[User:Themystic | Tom Aratyn]] (mystic)
** Displays informative messages to allow users to fix any invalidity in files or directories
** Displays an informative progress/status bar that informs users how far into the testing they are
*Needs proper source documentationThings that Alice and I discussed are reflected in this file:*Need to fix some of * Configuration checker*** yaml file validator*** paths.py validator**** checking the indentationpaths for existance***** notify user if path doesnt exist and ask user if they want it created**** checking if the directories have contents
</td>
<td valign="top">
*Main Contributor
** [[User:dwwoodsi|Dean Woodside]] (dean)
*With this script:
** Configuration of python framework is eased
</td>
<td valign="top">
**Captures <b>most</b> user errors through validation and saves user time and frustration to fix an error while running the framework
***Informs user of the directory/file invalidity or if they ran an invalid config file etc.
Additional:
*Users don't have to know the whole framework to configure it
** Documentation helps
*Users are not left staring at the console wondering what is happening when they run the Performance Tests
** Progress bar and informative messages
*Users don't have to waste hours/days/weeks debugging the code to find out why they are having problems configuring the framework
** I spent 2 weeks configuring the <strike>bloody</strike> framework
</td>
*[[#Extended_Progress_Chart_.28Version_2.29 | Version 2]]
*[[#Progress_.28Version_1.29 | Version 1]]
</td>
</tr>
<tr>
<td valign="top">
Firefox Performance Testing Framework Directory Structure
</td>
<td valign="top">
*Here is an overview of the Firefox Performance Testing Framework Directory Structure.
*The directory structure is revised
*There were too many extra libraries to load which caused too many dependencies
*Directory structure was difficult to keep track due to redundant directories and extra libraries
*All configuration is in one place!
** config.yaml
** paths.py
** constants
</td>
<td valign="top">
[[#Directory_Structure_of_Framework| Performance Testing Framework Directory Structure]]
</td>
</tr>
<tr>
<td valign="top">
Screen shots
</td>
<td valign="top">
*Here is a screen shot of the program when it is doing a performance testing:
[[Image:Perftesting running.png | Firefox Performance Testing running]]
</td>
<td valign="top">
Not Applicable
</td>
</tr>
<tr>
<td valign="top">
Next Steps
</td>
<td valign="top">
*Am I done? Not quite
</td>
<td valign="top">
[[#Next_Steps | Next Big Steps]]
</td>
</tr>
<tr>
<td valign="top">
Reflections
</td>
<td valign="top">
*Some thoughts
</td>
<td valign="top">
[[#Reflections_on_the_project | Reflections]]
</td>
</tr>
</table>
==Next Steps==
<table style="width:100%;" border="1px" valign="top" class="standard-table" cellpadding=2 cellspacing=0>
<th style="width: 15%; background-color:#333333; color:#F3F3F3">Task</th>
<th style="width: 30%; background-color:#333333; color:#F3F3F3">Details</th>
<th style="width: 10%; background-color:#333333; color:#F3F3F3">Priority</th>
<th style="width: 15%; background-color:#333333; color:#F3F3F3">Contributors</th>
<th style="width: 20%; background-color:#333333; color:#F3F3F3">Status</th>
<tr>
<td valign="top">Porting framework to other OSes</td>
<td valign="top">
* <b>What good is a performance testing framework if it only runs on Windows??</b>
* This has been a pending task for awhile.
* Since several students made a point that it should port to other OSes, I should look into that
* Students who use OS X, Unix Oses etc. couldn't test this framework
</td>
<td style="background-color: #990000; text-align: center; font-weight: bold; color: #ffffff;" valign="top">High</td>
<td valign="top">
* [[User:elichak|Liz Chak]]
</td>
<td valign="top">
<div style="width:100%;background-color:#F3F3F3;position:relative;margin-bottom:0px;">0% completed</div>
</td>
</tr>
<tr>
<td valign="top">Run both ts and tp at the same time</td>
<td valign="top">
*Will have a discussion with Alice Nodelman in regards to this
</td>
<td style="background-color: #E9DA73; text-align: center; font-weight: bold; color: #333333;" valign="top">Medium</td>
<td valign="top">
* [[User:elichak|Liz Chak]]
</td>
<td valign="top">
<div style="width:100%;background-color:#F3F3F3;position:relative;margin-bottom:0px;">0% completed</div>
</td>
</tr>
<tr>
<td valign="top">Create and execute new tests</td>
<td valign="top">
*New test case(s) for performance
*Will have a discussion with Alice Nodelman in regards to this
</td>
<td style="background-color: #E9DA73; text-align: center; font-weight: bold; color: #333333;" valign="top">Medium</td>
<td valign="top">
* [[User:elichak|Liz Chak]]
</td>
<td valign="top">
<div style="width:100%;background-color:#F3F3F3;position:relative;margin-bottom:0px;">0% completed</div>
</td>
</tr>
</table>
</td>
<td valign="top">
<b>Newer status</b>
The DPS909 class tested the documentation and framework out.
*Needed rework
<b>Older status</b>
*Improving the current documentation so that it's easier to follow
*Making sure that all the configuration documents are in one place
*This is done along with the code base work I'm doing
<div style="width:100%;background-color:#F3F3F3;position:relative;margin-bottom:0px;">90100% completed - Needs to be reviewed and tested out</div>
</td>
</tr>
*Scroll down and look for dom.allow_scripts_to_close_windows
*Double click on it to set it to true
=Directory Structure of Framework=
==Overview of Structure==
A glance at the Framework File Structure (CVS files not included):
<pre>
win32
|
|
|__ base_profile (dir)
| |
| |__ bookmarkbackups (dir)
| | |
| | |__ .html files
| |
| |__ Cache (dir)
| |
| |__ .bak, .html, .ini, .dat, .txt, .js, .rdf, .mfl files
|
|
|__ page_load_test(dir)
| |
| |__ base(dir)
| | |
| | |__ other dirs and .html files
| |
| |__ cycler.html & report.html
|
|
|__ startup_test
| |
| |__ startup_test.html
|
|
|__ extension_perf_reports (dir for generated reports)
|
|
|__ run_tests.py, paths.py, config.yaml and other .py, .html files
</pre>
*NOTE: Content in '''base_profile''' dir may vary
The following is written by Annie Sullivan (annie.sullivan@gmail.com):
==base_profile/==
*This directory contains the base profile used for testing.
*A copy of this profile is made for each testing profile, and extensions or prefs are added according to the test_configs array in run_tests.py.
*For the page load test to run correctly, the hostperm.1 file must be set to allow scheme:file uris to open in new windows, and the pref to force a window to open in a tab must not be set.
*The dom.allow_scripts_to_close_windows pref should also be set to true. The browser.shell.checkDefaultBrowser pref should be set to false.
==page_load_test/==
*This directory contains the JavaScript files and html data files for the page load test.
*The page load test opens a new window and cycles through loading each html file, timing each load.
==startup_test/==
*This directory contains the JavaScript to run the startup test.
*It measures how long it takes Firefox to start up.
==extension_perf_report/==
*This directory is where the generated report will go into.
*You may specify another directory to substitute this directory, but make sure that the '''paths.py''' is changed to point to it.
==run_tests.py, paths.py==
*These files should be configured to run the test on different machines, with different extensions or preferences. See setup above.
= Project News =
* [[davey_girl_comments| Erin Davey]]
* [[dwwoodsi perf comments| Dean Woodside]]
==Reflections on the project==
Now that the first phase of the project is done, I would like to sit back and reflect on some of the experiences I have had with this project. Not only did this project grow, I have been growing with it. I've learned endless amount of things by working on this project (not just technical stuff). And as the saying goes, "you only learn from experience"!
===Configuration frustration===
Words can't express how I was initially frustrated with the framework. I hit brick walls a countless number of times when I tried to set the framework up and running and I was on the verge of giving up. It was only through my own perseverance and determination that I got this to work.
Outcome: A list of things to fix on the framework to ease the configuration, strengthen the framework and better documentation!!
===First deliverable===
I asked the class to test my first deliverable. I had faith that my framework would bring delight to my testers, but I was proven otherwise. Even with the effort that I've put into my first deliverable, it still created a group of frustrated and agonized users. Many of them ranted on what could be improved on the framework, hence:
Outcome: A list of things to automate the framework and EASE THE CONFIGURATION
===Final deliverable===
I'm glad that my first tester was thrilled about my Performance Testing. There is nothing better than a happy user. What I've learned is that TESTING is the KEY to a successful application. Why do so many of us developers neglect that??
===Credits===
[[User:dwwoodsi|Dean Woodside]], Alice Nodelman, [[User:Bhearsum | Ben Hearsum]], [[User:Mylau | Michael Lau]], [[User:Eor | Eva Or]], [[User:djhamp-g | David Hamp Gonsalves ]], [[User:David.humphrey|Dave Humphrey]], [[User:robcee|Rob Campbell]] and of course, the DPS909 class who tested my framework. These are the individuals who played a significant part in making this framework headed toward success!
==Bon Echo Community Test Day==
