3
edits
Changes
m
Alice Nodelman
*Discussion on the things that need to be fixed to improve and strengthen the framework
<b>==In class Firefox Performance Testing=='''Before you begin'''# You have to be on a Windows operating system# Take a deep breath and go through the [[Firefox_Performance_Testing_:_A_Python_framework_for_Windows_Documentation | Performance Testing Setup Configuration Documentation]]# Make a page under [[Firefox_Performance_Testing_:_A_Python_framework_for_Windows#Comments_on_the_Firefox_Performance_Testing_Framework | Comments on the Firefox Performance Testing Framework (see below)]] to list out your comments on the framework# The performance testing is only successful if you see generated results in the reports directory (you'll understand what this means once you start)# Ask [[User:Elichak | Liz Chak]] if you have any questions '''Things to look out for'''# If the documentation is easy to follow# Did you stumble upon any difficulty? If so, state them# List any system errors, if any (with the error message and a brief description of what you think caused it) ===Comments on the Firefox Performance Testing Framework==='''Instructions:'''* Make a page starting with your name/nick and end with "perf comments" (to ensure uniqueness)* Example: [[Liz Chak perf comments]] '''List of comments:''' * [[Mark D'Souza perf comments]]* [[Sherman Fernandes perf comments]]* [[Aditya Nanda Kuswanto perf comments]]* [[Richard Chu perf comments]]* [[David Manley perf comments]]* [[Djhamp-g perf comments]]* [[Colin Guy perf comments]]* [[Mohamed Attar perf comments]]* [[Man Choi Kwan perf comments]]* [[Mark Paruzel pref comments]]* [[Jeff Mossop perf comments]]* [[Melissa Peh perf comments]]* [[Paul Yanchun Gu comments]]* [[Vanessa Miranda comments]]* [[Philip Vitorino perf comments]]* [[Paul St-Denis Moe Bagheri perf comments]]* [[Cesar Oliveira perf comments]]* [[Firefox Performance Testing : A Python framework for Windows:Comments/Ben Hearsum|Ben Hearsum]]* [[Themystic_perf_comments| Tom Aratyn]]* [[davey_girl_comments| Erin Davey]]* [[dwwoodsi perf comments| Dean Woodside]] ==Reflections on the project==Now that the first phase of the project is done, I would like to sit back and reflect on some of the experiences I have had with this project. Not only did this project grow, I have been growing with it. I've learned endless amount of things by working on this project (not just technical stuff). And as the saying goes, "you only learn from experience"! ===Configuration frustration===Words can't express how I was initially frustrated with the framework. I hit brick walls a countless number of times when I tried to set the framework up and running and I was on the verge of giving up. It was only through my own perseverance and determination that I got this to work. Outcome: A list of things to fix on the framework to ease the configuration, strengthen the framework and better documentation!! ===First deliverable===I asked the class to test my first deliverable. I had faith that my framework would bring delight to my testers, but I was proven otherwise. Even with the effort that I've put into my first deliverable, it still created a group of frustrated and agonized users. Many of them ranted on what could be improved on the framework, hence: Outcome: A list of things to automate the framework and EASE THE CONFIGURATION ===Final deliverable===I'm glad that my first tester was thrilled about my Performance Testing. There is nothing better than a happy user. What I've learned is that TESTING is the KEY to a successful application. Why do so many of us developers neglect that?? ===Credits===[[User:dwwoodsi|Dean Woodside]], Alice Nodelman, [[User:Bhearsum | Ben Hearsum]], [[User:Mylau | Michael Lau]], [[User:Eor | Eva Or]], [[User:djhamp-g | David Hamp Gonsalves ]], [[User:David.humphrey|Dave Humphrey]], [[User:robcee|Rob Campbell]] and of course, the DPS909 class who tested my framework. These are the individuals who played a significant part in making this framework headed toward success! ==Bon Echo Community Test Day</b>==
= Project Contributor(s) =
[[User:dwwoodsi|Dean Woodside]] (dean)
* Submitted an sh file that automates the tedious performance testing framework configuration
Alice Nodelman
*Discussed on the things that need to be fixed to improve and strengthen the framework
*Gave suggestions on the new Performance Testing framework
[[User:Bhearsum | Ben Hearsum]] (bhearsum)
*Set up the VM for performance testing
*Helped get me started with the debugging process for report.py, run_tests.py and ts.py [[User:Mylau | Michael Lau]] (mylau)*Added comments on the documentation for setting up Performance Testing framework for Windows*Tested the new and improved Performance Testing framework (version 1)*Gave constructive feedback on documentation (version 1) [[User:Eor | Eva Or]] (eor)*Tested the new and improved Performance Testing framework (version 1)*Gave constructive feedback on new documentation (version 1) [[User:djhamp-g | David Hamp Gonsalves ]] (inveigle)*Gave pointers on flushing buffer*Helped with some grammar and sentence structuring for documentation (version 1)*Tested and gave constructive feedback on the framework (version 1)*Looked into a batch file for automating configuration, gave pointers*Tested and commented on the new framework (version 2) [[User: robcee | Rob Campbell]] (robcee)*Gave me a number of python tips :)
[[User:Themystic | Tom Aratyn]] (mystic)
*Introduced Closures in Python
==In-Class Contributors==Please let me know if I missed you out. I've only listed the people whom I've received comments from. Those of you who participated but isn't listed as an in-class contributor, please list your comments [[Firefox_Performance_Testing_:_A_Python_framework_for_Windows#In_class_Firefox_Performance_Testing | here]]* [[User:Mylau Mdsouza| Michael LauMark D'Souza]] (mylaumdsouza)*Added comments on the documentation for setting up Performance Testing framework for Windows[[User:Sherman|Sherman Fernandes]] (sjfern)* [[User:Ankuswan|Aditya Nanda Kuswanto]] (vipers101)* [[User:dmmanley|Dave Manley]] (seneManley)* [[User:Cbguy1|Colin Guy]] (Guiness)* [[User:mattar1|Mohamed Attar]] (mojo)* [[User:mckwan|Man Choi Kwan]] (mckwan)* [[User:RealMarkP|Mark Paruzel]] (RealMarkP or FakeMarkP)* [[User:jbmossop|Jeff Mossop]] (JBmossop)* [[User:melz|Melissa Peh]] (melz)* [[User:Paul|Paul Yanchun Gu]] (gpaul)* [[User:Vpmirand|Vanessa Miranda]] (vanessa)* [[User:Pcvitori|Philip Vitorino]] (philly)* [[User:PStD|Paul St-Denis]] (pstdenis)* [[User:moe|Mohammad Tirtashi]] (moe)* [[User:Cdolivei|Cesar Oliveira]] (cesar)* [[User:Andrew|Andrew Smith]] (andrew)* [[User:Bhearsum | Ben Hearsum]] (bhearsum)* [[User:Eldavey|Erin Davey]] (davey_girl)* [[User:themystic| Tom Aratyn]] (mystic)* [[User:djhamp-g | David Hamp]]* [[User:dwwoodsi|Dean Woodside]] (dean)
= Project Details =
==Improved Documentation==
===Latest===
[[New_Firefox_Performance_Testing_Documentation | New Firefox Performance Testing Documentation]]
===First Attempt===
[[Firefox_Performance_Testing_:_A_Python_framework_for_Windows_Documentation | Performance Testing Setup Configuration Documentation]]
==Details==
*Second, it measures performance characteristics while it's running the pageload tests--you can track cpu speed, memory, or any of the other counters listed [http://technet2.microsoft.com/WindowsServer/en/Library/86b5d116-6fb3-427b-af8c-9077162125fe1033.mspx?mfr=true here].
==Web logs==*[http://weblogs.mozillazine.org/qa/ Mozilla Quality Assurance and Testing Blog for Mozilla Firefox and Thunderbird]*[http://wiki.mozilla.org/MozillaQualityAssurance:Performance_Testing Mozilla Quality Assurance Performance Testing] ==Submission==Still in progress!<br />While it's sizzling, try out the Firefox Performance Testing Framework (version 2) at [[New_Firefox_Performance_Testing_Documentation#Quick_start| New Firefox Performance Testing Documentation Quick Start section]] - 3 easy steps<table style="width:100%;" border="1px" valign="top" class="standard-table" cellpadding=2 cellspacing=0><th style="width: 20%; background-color:#333333; color:#F3F3F3">Artifact</th><th style="width: 30%%; background-color:#333333; color:#F3F3F3">Details</th><th style="width: 20%; background-color:#333333; color:#F3F3F3">Links</th><tr><td valign="top">run_tests.py</td><td valign="top">*Python file used to execute the Firefox Performance Testing*Some of the things it does:** Validates files and directories needed for Performance Testing** Displays informative messages to allow users to fix any invalidity in files or directories** Displays an informative progress/status bar that informs users how far into the testing they are*Things that Alice and I discussed are reflected in this file:** Configuration checker*** yaml file validator*** paths.py validator**** checking the paths for existance***** notify user if path doesnt exist and ask user if they want it created**** checking if the directories have contents </td><td valign="top">[[run_tests.py]]</td></tr><tr><td valign="top">perfconfig.sh</td><td valign="top">This script is used to automate the tedious Firefox Performance Testing Configuration*Main Contributor** [[User:dwwoodsi|Dean Woodside]] (dean)*With this script:** Configuration of python framework is eased </td><td valign="top">[[perfconfig.sh]]</td></tr><tr><td valign="top">Progress bar class</td><td valign="top">*Modified open source code to be used for this project*Displays a progress/status bar so users will know at which stage of the Performance Testing they are at (by percentage%, sort of)</td><td valign="top">[[pb.py]]</td></tr><tr><td valign="top">paths.py</td><td valign="top">*This is the file that contains all the paths that are needed for the Performance Testing*Fixed the paths and modified some documentation</td><td valign="top">[[paths.py]]</td></tr><tr><td valign="top">Firefox Performance Testing Documentation (Version 1)</td><td valign="top">*This was the first documentation that was written to improve the documentation provided in the initialFirefox Performance Testing Framework (Refer to [[Readme.txt]])*It was deemed to be too elaborative and needed to be more concise*Needed work*Users didn't like the configuration steps they had to take** For more ranting:[[#Comments_on_the_Firefox_Performance_Testing_Framework | DPS909 Student comments]]</td><td valign="top">[[Firefox_Performance_Testing_:_A_Python_framework_for_Windows_Documentation | Performance Testing Setup Configuration Documentation]]</td></tr><tr><td valign="top">Firefox Performance Testing Documentation (Version 2)</td><td valign="top">*<b>To try Firefox Performance Testing out, go to this documentation</b>*This the new documentation that is improved with the [[#Comments_on_the_Firefox_Performance_Testing_Framework | student comments]]*Building a user base.. some comments on this NEW framework**[[New Firefox Performance Testing Framework comments]] <b>Goals achieved:</b>* Effective**Easy-to-use**Free from confusion**Informative and helpful*Efficient**Takes almost no time to set up the framework**Captures <b>most</b> user errors through validation and saves user time and frustration to fix an error while running the framework***Informs user of the directory/file invalidity or if they ran an invalid config file etc. Additional:*Users don't have to know the whole framework to configure it** Documentation helps*Users are not left staring at the console wondering what is happening when they run the Performance Tests** Progress bar and informative messages*Users don't have to waste hours/days/weeks debugging the code to find out why they are having problems configuring the framework** I spent 2 weeks configuring the <strike>bloody</strike> framework </td><td valign="top">[[New_Firefox_Performance_Testing_Documentation | New Firefox Performance Testing Documentation]]</td></tr> <tr><td valign="top">Status Documentation</td><td valign="top">*Here are the progress charts for this project*They are divided into 3:** Performance Testing Framework First Version** Performance Testing Framework Second Version** More details of things to fix for the framework </td><td valign="top">*[[Things to fix for Performance Testing Framework | More details of things to fix for Performance Testing Framework]]*[[#Extended_Progress_Chart_.28Version_2.29 | Version 2]]*[[#Progress_.28Version_1.29 | Version 1]]</td></tr><tr><td valign="top">Firefox Performance Testing Framework Directory Structure</td><td valign="top">*Here is an overview of the Firefox Performance Testing Framework Directory Structure. *The directory structure is revised*There were too many extra libraries to load which caused too many dependencies*Directory structure was difficult to keep track due to redundant directories and extra libraries *All configuration is in one place!** config.yaml** paths.py** constants</td><td valign="top">[[#Directory_Structure_of_Framework| Performance Testing Framework Directory Structure]]</td></tr><tr><td valign="top">Screen shots</td><td valign="top">*Here is a screen shot of the program when it is doing a performance testing: [[Image:Perftesting running.png | Firefox Performance Testing running]] </td><td valign="top">Not Applicable</td></tr><tr><td valign="top">Next Steps</td><td valign="top">*Am I done? Not quite</td><td valign="top">[[#Next_Steps | Next Big Steps]]</td></tr><tr><td valign="top">Reflections</td><td valign="top">*Some thoughts</td><td valign="top">[[#Reflections_on_the_project | Reflections]]</td></tr></table> ==Next Steps==<table style="width:100%;" border="1px" valign="top" class="standard-table" cellpadding=2 cellspacing=0><th style="width: 15%; background-color:#333333; color:#F3F3F3">Task</th><th style="width: 30%; background-color:#333333; color:#F3F3F3">Details</th><th style="width: 10%; background-color:#333333; color:#F3F3F3">Priority</th><th style="width: 15%; background-color:#333333; color:#F3F3F3">Contributors</th><th style="width: 20%; background-color:#333333; color:#F3F3F3">Status</th><tr><td valign="top">Porting framework to other OSes</td><td valign="top">* <b>What good is a performance testing framework if it only runs on Windows??</b>* This has been a pending task for awhile. * Since several students made a point that it should port to other OSes, I should look into that* Students who use OS X, Unix Oses etc. couldn't test this framework</td><td style="background-color: #990000; text-align: center; font-weight: bold; color: #ffffff;" valign="top">High</td><td valign="top">* [[User:elichak|Liz Chak]]</td><td valign="top"> <div style="width:100%;background-color:#F3F3F3;position:relative;margin-bottom:0px;">0% completed</div></td></tr><tr><td valign="top">Run both ts and tp at the same time</td><td valign="top">*Will have a discussion with Alice Nodelman in regards to this</td> <td style="background-color: #E9DA73; text-align: center; font-weight: bold; color: #333333;" valign="top">Medium</td><td valign="top">* [[User:elichak|Liz Chak]]</td><td valign="top"><div style="width:100%;background-color:#F3F3F3;position:relative;margin-bottom:0px;">0% completed</div></td></tr> <tr><td valign="top">Create and execute new tests</td><td valign="top">*New test case(s) for performance*Will have a discussion with Alice Nodelman in regards to this</td><td style="background-color: #E9DA73; text-align: center; font-weight: bold; color: #333333;" valign="top">Medium</td><td valign="top">* [[User:elichak|Liz Chak]]</td><td valign="top"><div style="width:100%;background-color:#F3F3F3;position:relative;margin-bottom:0px;">0% completed</div></td></tr> </table> ==Extended Progress Chart (Version 2)==<table style="width:100%;" border="1px" valign="top" class="standard-table" cellpadding=2 cellspacing=0><th style="width: 15%; background-color:#333333; color:#F3F3F3">Task</th><th style="width: 30%; background-color:#333333; color:#F3F3F3">Details</th><th style="width: 10%; background-color:#333333; color:#F3F3F3">Priority</th><th style="width: 15%; background-color:#333333; color:#F3F3F3">Contributors</th><th style="width: 20%; background-color:#333333; color:#F3F3F3">Status</th><tr><td valign="top">Read all student comments about framework and improve it</td><td valign="top">* On Nov 29th, students of DPS909 contributed by testing the framework* They have listed constructive feedback and many great suggestions to help improve the setup of the framework** [[Firefox_Performance_Testing_:_A_Python_framework_for_Windows#Comments_on_the_Firefox_Performance_Testing_Framework | Student comments about framework]] </td><td style="background-color: #990000; text-align: center; font-weight: bold; color: #ffffff;" valign="top">High</td><td valign="top">* [[User:elichak|Liz Chak]]</td><td valign="top">* I've read some comments but I need some time to drill down ones that will help to improve the framework.* Nevertheless, all the comments are really REALLY helpful to facilitate in improving this framework. <div style="width:100%;background-color:#F3F3F3;position:relative;margin-bottom:0px;">100% completed</div></td></tr><tr><td valign="top">Drill and narrow down the student comments to improve framework</td><td valign="top">*Improve framework as per suggested by the students: [[Firefox_Performance_Testing_:_A_Python_framework_for_Windows#Comments_on_the_Firefox_Performance_Testing_Framework | Student comments about framework]] *Only pick the important ones*[[Things to fix for Performance Testing Framework | More details]]</td><td style="background-color: #990000; text-align: center; font-weight: bold; color: #ffffff;" valign="top">High</td><td valign="top">* [[User:elichak|Liz Chak]]</td><td valign="top">* Working on some of the comments that helps make the framework stronger and more robust* Some tasks might be redundant. Nonetheless, I'm clear about what should be fixed<div style="width:100%;background-color:#F3F3F3;position:relative;margin-bottom:0px;">In progress</div></td></tr><tr><td valign="top">Fix [[Firefox_Performance_Testing_:_A_Python_framework_for_Windows_Documentation | Performance Testing Setup Configuration Documentation]] and framework configuration</td><td valign="top">* From the [[Firefox_Performance_Testing_:_A_Python_framework_for_Windows#Comments_on_the_Firefox_Performance_Testing_Framework | student comments about the framework]], I have gathered that the documentation have to be fixed as some parts are vague and some important areas are not highlighted. * Some Items ([[Things to fix for Performance Testing Framework | More details]]):** Make the ''Create extension_perf_reports directory''' part bold*** Maybe ask the user if they want it to be generated if it doesn't exist (will be done in framework)*** Should it be created from the sh script??** '''config.yaml''' file*** If user chooses to test with no preferences or extensions, should mention about commenting the lines beneath them.** Additional details regarding Framework Prerequisites would reduce confusion. *** Comment from student: There isn't any harm in "dumbing it down" a level. *[[Things to fix for Performance Testing Framework | More details]] </td><td style="background-color: #990000; text-align: center; font-weight: bold; color: #ffffff;" valign="top">High</td><td valign="top">* [[User:elichak|Liz Chak]]</td><td valign="top"><b>New status</b> [[New Firefox Performance Testing Documentation]] <b>Older status</b> Will start on this after drilling down the [[Firefox_Performance_Testing_:_A_Python_framework_for_Windows#Comments_on_the_Firefox_Performance_Testing_Framework | student comments]] <div style="width:100%;background-color:#F3F3F3;position:relative;margin-bottom:0px;">In progress</div></td></tr><tr><td valign="top">Pre-requisites: Python, Cywin and .dll Installation Guidelines</td><td valign="top">* Order of installations* Cygwin**Students were confused as to whether they should keep the default [http://www.cygwin.com/setup.exe Cygwin] setup/packages as the documentation didn't outline that.**They inquired which Cygwin packages are required* Concerns about where the .[http://www.driverskit.com/dll/msvcp71.dll/2371.html msvcp71.dll] file should be installed after it's downloaded * A few students got the error: ImportError: No module named win32pdh ** missed out install the Python Win32 Extensions - sh script handles it now* Now that the sh script handles the pre-requisite installation, are the following necessary?** Some files seem to have dependencies on another. Need to point whether some files have to be installed in order.** Some files have the same installation GUI, which makes it hard to track down which have been installed and which have not. Some screenshots may help.** For '''Cygwin''' installation, should note that default package setting is sufficient. Remove all doubts.</td><td style="background-color: #E9DA73; text-align: center; font-weight: bold; color: #333333;" valign="top">Medium</td><td valign="top">* [[User:elichak|Liz Chak]]</td><td valign="top">*All these issues are mitigated by the [[perfconfig.sh]] script.*This script automates all the tedious steps you have to go through to run your performance testing which was, unfortunately, loathed by users. (Based on [[#Comments_on_the_Firefox_Performance_Testing_Framework | student comments]])*See it for yourself! Try it out: [[New_Firefox_Performance_Testing_Documentation#Quick_start| New Firefox Performance Testing Documentation Quick Start section]]*I'm also gathering user feedback: [[New Firefox Performance Testing Framework comments]]<div style="width:100%;background-color:#F3F3F3;position:relative;margin-bottom:0px;">100% done</div></td></tr><tr><td valign="top">Configuring Environment Specifications in documentation</td><td valign="top"><b>BASE_PROFILE_DIR</b>* No instructions on what exactly to place in the Hostperm.1 file.* Hostperm.1 file is autogenerated and warns against editing it.* Mention how to modify the Hostperm.1 file.</td><td style="background-color: #E9DA73; text-align: center; font-weight: bold; color: #333333;" valign="top">Medium</td><td valign="top">* [[User:elichak|Liz Chak]]</td><td valign="top">I have nothing to work with besides the [[Readme.txt | README.TXT]] file by Annie Sullivan. Therefore, I have to consult Alice Nodelman in this area.<div style="width:100%;background-color:#F3F3F3;position:relative;margin-bottom:0px;">In progress</div></td></tr> <tr><td valign="top">Automate environment configuration</td><td valign="top"> *One of the [[Firefox_Performance_Testing_:_A_Python_framework_for_Windows#Comments_on_the_Firefox_Performance_Testing_Framework | student comments]] is to write a bat/sh file to automate the configurations of the pre-requisites for the framework</td><td style="background-color: #E9DA73; text-align: center; font-weight: bold; color: #333333;" valign="top">Medium</td><td valign="top">* [[User:dwwoodsi|Dean Woodside]] (dean)** Submitted sh script* [[User:djhamp-g | David Hamp-Gonsalves]] (inveigle)** Looked into this, gave pointers</td><td valign="top">* [[User:dwwoodsi|Dean Woodside]] has submitted a sh scripting for this. It works great. Needs to be touched up**[http://www.yorku.ca/abel/liz/mozilla/perfconfig.zip Sh script zip] * [[User:elichak |Liz Chak]] will fix up the sh script**Doesn't make a reports folder (should this be in the sh script??)**Still runs the whole script even if it couldn't download the pre-requisites, should terminate**Missed out the paths.py file**Requires touch ups, but everything else is good**Proper documentation Files:*[[perfconfig.sh]] - Source code*[http://www.yorku.ca/abel/liz/mozilla/perfconfig.sh Actual Script to configure Firefox Performance Testing Framework]<div style="width:100%;background-color:#F3F3F3;position:relative;margin-bottom:0px;">100% completed</div></td></tr> <tr><td valign="top">Porting framework to other OSes</td><td valign="top">* This has been a pending task for awhile. * Since several students made a point that it should port to other OSes, I should look into that* Students who use OS X, Unix Oses etc. couldn't test this framework</td><td style="background-color: #E9DA73; text-align: center; font-weight: bold; color: #333333;" valign="top">Medium</td><td valign="top">* [[User:elichak|Liz Chak]]</td><td valign="top"> <div style="width:100%;background-color:#F3F3F3;position:relative;margin-bottom:0px;">0% completed</div></td></tr></table> ==Progress(Version 1)==
<table style="width:100%;" border="1px" valign="top" class="standard-table" cellpadding=2 cellspacing=0>
<th style="width: 15%; background-color:#333333; color:#F3F3F3">Status</th>
<tr>
<td valign="top">[[Firefox_Performance_Testing_:_A_Python_framework_for_Windows_Documentation | Performance Testing Setup Configuration Documentation]]</td>
<td valign="top">
*The current setup configuration documentations are in text files and are very hard to follow.
</td>
<td valign="top">
<b>Newer status</b>
The DPS909 class tested the documentation and framework out.
*Needed rework
<b>Older status</b>
*Improving the current documentation so that it's easier to follow
*Making sure that all the configuration documents are in one place
*This is done along with the code base work I'm doing
<div style="width:100%;background-color:#F3F3F3;position:relative;margin-bottom:0px;">40100% completed</div>
</td>
</tr>
</td>
<td valign="top">
<div style="width:100%;background-color:#F3F3F3;position:relative;margin-bottom:0px;">Startedand ongoing</div>
</td>
</tr>
<div style="width:100%;background-color:#F3F3F3">98100% completed (Alice will let me know if I'm missing something - therefore the 2% is reserved for thatreviewed)</div>
</td>
</tr>
* Alice Nodelman
** Gave pointers
* [[User:Robcee | Rob Campbell]]
** Gave some suggestions in relation to Python
</td>
<td valign="top">
<b>Most updated progress</b>
*Working state
** In run_tests.py, it checks all the directory paths in paths.py (paths.CYGWIN, paths.SYNC, paths.BASE_PROFILE_DIR, paths.REPORTS_DIR)
** Prints to console each time it's checking to allow the developer to know which dir it's checking
** If a directory doesn't exist, the app will terminate and will let the developer know which directory has to be made
** Alternative, change the path in paths.py to point to the right path
** Initially I have it set to make the directories if they don't exist, however, Alice adviced that I should allow the developer to do it himself/herself
** Problem Checks if base_profile contents exist!** Got some suggestions from [[User:Robcee | Rob Campbell]], going to work with checking files in this format (his suggestions** Checks file:///c:/)urls
*Things left to do
** Check if base_profile contents exist- DONE*** Alice adviced that I should check contents in general - no specific files or directories because they can change depending on what the developer is testing** Check files in (file:///c:/) format - DONE*** create a temporary variable for the local pathname, convert the path to the os path and then check*** split and join*** urlparse** Work on checking Cygwin paths *** split and join
*I have fixed the program to check for the existence of the base_profile dir and it also checks if the bookmarkbackups, Cache dirs and perf.js dir exist.
<div style="width:100%;background-color:#F3F3F3">85100% completed</div>
</td>
</td>
<td valign="top">
<div style="width:100%;background-color:#F3F3F3;position:relative;margin-bottom:0px;">0Changed structure of directory 100% completed- needs to be reviewed</div>
</td>
</tr>
<td valign="top">Get class to test out the Performance Testing framework</td>
<td valign="top">
The improved framework has to be tested to get constructive feedback from the users.using the new [[Firefox_Performance_Testing_:_A_Python_framework_for_Windows_Documentation | Performance Testing Setup Configuration Documentation]]
</td>
<td style="background-color: #F8F8F8; text-align: center; font-weight: bold; color: #333333;" valign="top">Low</td>
<td valign="top">
* [[User:elichak|Liz Chak]]
* [[User:eor | Eva Or]] & [[User:mylau | Mike Lau]]
* [[Firefox_Performance_Testing_:_A_Python_framework_for_Windows#In-Class_Contributors|Whole class]]!
** Setup framework using the new documentation - Things to look out for: If it's user-oriented and if it is easy to setup
** Gave constructive feedback on documentation
</td>
<td valign="top">
<div style="width:100%;background-color:#F3F3F3;position:relative;margin-bottom:0px;">0100% completed</div>
</td>
</tr>
</table>
==Comments on the current [[Readme.txt | README.TXT]] Documentation (By Mike Lau)==
'''Getting Started'''
'''Comments on The [[Readme.txt | README.txt TXT]] File* The [[Readme.txt | README.txt TXT]] file is hard to read in notepad. There's horizontal scrolling after opening the file.
* The file should have output samples to show the user what output to expect and to ensure they are on the right track.
* The pre-requisites part of the [[Readme.txt | README .TXT]] file is missing minimum size to be allocated in the computer to complete the install.
* Following the procedures for installation was difficult. The document should number the procedures and have sample outputs to ensure the user is going on the right track
* Some of the folders which was listed in the Directory Structure was missing in the setup procedures. The Directory Structure should be used as a guide to ensure users have the right folders in place. Also, there should be an image to represent the directory structure. Some of the folders which needed to be created on top of following the procedures provided were: base_profile, extension_perf_testing, extension_perf_reports
==Solution: Firefox doesn't know how to open this address, because the protocol (c) isn't associated with any programs==
In paths.py, the paths for INIT_URL, TS_URL and TP_URL have to be in this format a local file url, not file path (file:///c:/):
<pre>
"""The path to the file url to load when initializing a new profile"""
* By default in paths.py: BASE_PROFILE_DIR = r'C:\extension_perf_testing\base_profile'
* The BASE_PROFILE_DIR could be a different path, it doesn't have to be like the one above.
==Problem: This page should close Firefox. If it does not, please make sure that the dom.allow_scripts_to_close_windows preference is set to true in about:config==
*You have to change the dom.allow_scripts_to_close_windows preference to true
==Solution: This page should close Firefox. If it does not, please make sure that the dom.allow_scripts_to_close_windows preference is set to true in about:config==
*Type in about:config in the address bar
*Scroll down and look for dom.allow_scripts_to_close_windows
*Double click on it to set it to true
=Directory Structure of Framework=
==Overview of Structure==
A glance at the Framework File Structure (CVS files not included):
<pre>
win32
|
|
|__ base_profile (dir)
| |
| |__ bookmarkbackups (dir)
| | |
| | |__ .html files
| |
| |__ Cache (dir)
| |
| |__ .bak, .html, .ini, .dat, .txt, .js, .rdf, .mfl files
|
|
|__ page_load_test(dir)
| |
| |__ base(dir)
| | |
| | |__ other dirs and .html files
| |
| |__ cycler.html & report.html
|
|
|__ startup_test
| |
| |__ startup_test.html
|
|
|__ extension_perf_reports (dir for generated reports)
|
|
|__ run_tests.py, paths.py, config.yaml and other .py, .html files
</pre>
*NOTE: Content in '''base_profile''' dir may vary
The following is written by Annie Sullivan (annie.sullivan@gmail.com):
==base_profile/==
*This directory contains the base profile used for testing.
*A copy of this profile is made for each testing profile, and extensions or prefs are added according to the test_configs array in run_tests.py.
*For the page load test to run correctly, the hostperm.1 file must be set to allow scheme:file uris to open in new windows, and the pref to force a window to open in a tab must not be set.
*The dom.allow_scripts_to_close_windows pref should also be set to true. The browser.shell.checkDefaultBrowser pref should be set to false.
==page_load_test/==
*This directory contains the JavaScript files and html data files for the page load test.
*The page load test opens a new window and cycles through loading each html file, timing each load.
==startup_test/==
*This directory contains the JavaScript to run the startup test.
*It measures how long it takes Firefox to start up.
==extension_perf_report/==
*This directory is where the generated report will go into.
*You may specify another directory to substitute this directory, but make sure that the '''paths.py''' is changed to point to it.
==run_tests.py, paths.py==
*These files should be configured to run the test on different machines, with different extensions or preferences. See setup above.
= Project News =
==Tuesday, 21 Nov 2006==
Refer to progress chart.
[[Firefox_Performance_Testing_:_A_Python_framework_for_Windows#ProgressProgress_.28Version_1.29| Performance Testing Framework progress chart]] ==Wednesday, 29 Nov 2006==In-class Performance Testing Framework Configuration* DPS909 students contributed by testing the framework* Students have listed constructive feedback and many great suggestions to help improve the setup of the framework** [[Firefox_Performance_Testing_:_A_Python_framework_for_Windows#Comments_on_the_Firefox_Performance_Testing_Framework | Student comments about framework]] ==Sunday, 3 Dec 2006==* [[User:dwwoodsi|Dean Woodside]] has submitted a sh scripting for this. It works great. Needs to be touched up.**[http://www.yorku.ca/abel/mozilla/perfconfig.zip Sh script zip] * [[User:elichak |Liz Chak]] will fix up the sh script**Doesn't make a reports folder **Still runs the whole script even if it couldn't download the pre-requisites, should terminate**Missed out the paths.py file**Requires touch ups, but everything else is good ==Sunday, 10 Dec 2006==* Narrowing down the things that need to be done before 13 December 2006* Refer to [[#Extended_Progress_Chart_.28Version_2.29 | Extended Progress Chart]] ==Wednesday, 13 Dec 2006==* Created a [[New_Firefox_Performance_Testing_Documentation | new documentation]]* Finished my parts for the Performance Testing Framework** Refer to [[Firefox_Performance_Testing_:_A_Python_framework_for_Windows#Submission | submission]] The current Firefox Performance Testing Framework is effective and efficient. *<b>To try Firefox Performance Testing out, go to this documentation</b>*This the new documentation that is improved with the [[#Comments_on_the_Firefox_Performance_Testing_Framework | student comments]]*Building a user base.. some comments on this NEW framework**[[New Firefox Performance Testing Framework comments]] <b>Goals achieved:</b>* Effective**Easy-to-use**Free from confusion**Informative and helpful*Efficient**Takes almost no time to set up the framework**Captures <b>most</b> user errors through validation and saves user time and frustration to fix an error while running the framework***Informs user of the directory/file invalidity or if they ran an invalid config file etc. Things To-do (Immediately)*Clean up this page - almost done*Document source - almost done*Fix up the [[New_Firefox_Performance_Testing_Documentation | Performance Testing Documentation]]
=Project References=
*I recommend this site these sites to learn about python:
**[http://aspn.activestate.com/ASPN/Cookbook/Python ASPN Python Cookbook]
**[http://docs.python.org/lib/os-file-dir.html Files and Directories]
= Project Events =
:Friday, October 06, 2006, from 7am - 5pm PDT
:[http://wiki.mozilla.org/Mozilla_QA_Community:BonEcho_2.0RC1prerelease_Community_Test_Day Mozilla QA Community:BonEcho 2.0RC1 prerelease Community Test Day ]
