Difference between revisions of "Unit Testing"
(→Unit testing log) |
(→What I have learned) |
||
(15 intermediate revisions by the same user not shown) | |||
Line 194: | Line 194: | ||
− | ==== Step 2. | + | ==== Step 2. Write a test suite variable called suite that holds all your test cases. ==== |
The following is example of bookmarks test suite. | The following is example of bookmarks test suite. | ||
Line 285: | Line 285: | ||
=== Customize Extension Menu Item Name === | === Customize Extension Menu Item Name === | ||
− | If users want to change the menu item name, they can open file unittesting.dtd under folder local. | + | If users want to change the menu item name for a specific test case, they can open file '''unittesting.dtd''' under folder '''local'''. |
[[image:extension02.png|600px]] <br /> | [[image:extension02.png|600px]] <br /> | ||
− | And then they can change the first item in the file unittesting.dtd. | + | And then they can change the first item to your test case name in the file unittesting.dtd. |
[[image:extension03.png|600px]] <br /> | [[image:extension03.png|600px]] <br /> | ||
Line 296: | Line 296: | ||
=== Run the tests === | === Run the tests === | ||
− | After adding the test cases to the extension, the participant can install the extension by drag and drop the extension into Firefox, and then click on install to install the unit testing framework with your test cases. | + | After adding the test cases to the extension, the participant can install the extension by drag and drop the extension into Firefox, and then click on install to install the unit testing framework with your test cases. |
+ | |||
+ | Running test cases is also easy, user can click on '''Tools''' menu from Firefox, and you will see an extension called '''Unit Testing for [...]'''. | ||
[[image:extension01.png|602px]] <br /> | [[image:extension01.png|602px]] <br /> | ||
− | Select the Unit Testing, system will pop up the following testing window. | + | Select the '''Unit Testing for [...]''', system will pop up the following testing window. |
[[image:extension04.png|602px]] <br /> | [[image:extension04.png|602px]] <br /> | ||
Line 318: | Line 320: | ||
=== Unit testing log === | === Unit testing log === | ||
− | Right now, this framework provides a detail log | + | Right now, this framework provides a detail log for test running. This will allow users to track the detail testing process and any error that encountered during the testing. |
− | In order to | + | In order to check the error console, go to '''Tools''' menu in Firefox, and select '''Error Console'''. |
[[image:extension05.png|602px]] <br /> | [[image:extension05.png|602px]] <br /> | ||
− | After push the button, the testing process will be displayed in console. | + | After push the '''Run''' button, the testing process message will be displayed in console. |
[[image:extension06.png|602px]] <br /> | [[image:extension06.png|602px]] <br /> | ||
Line 338: | Line 340: | ||
[[image:extension06-2.png|602px]] <br /> | [[image:extension06-2.png|602px]] <br /> | ||
− | |||
{{clear}} | {{clear}} | ||
Line 359: | Line 360: | ||
=== What I have learned === | === What I have learned === | ||
− | Based on my experice in open source developing, I had a lot of fun for doing it. Also I'd like to address some improtant comments when | + | Based on my experice in open source developing, I had a lot of fun for doing it. Also I'd like to address some improtant comments when I was working on the open source. |
+ | * It was very time-consuming. Confidence and patience are the key to success. | ||
+ | : This project has no prototype or even any example; most of time I couldn't get help because very less people work on this; and different people will do a different way in unit testing. It took me a lot of time to do researching, trying, and debugging. It also needs a lot of patience; if one way doesn't work, you still need to keep working on it until find the solution. You can take a short break and a deep breath, but never give up. | ||
* Documentation. | * Documentation. | ||
: Any code we wrote should be documentated. It's very important for you to check back and for the people who want to contrubute to it.This is really important in open source community. Because I couldn't find the documentation for the Bookmarks services, I cannot make the bookmark unit testing work. | : Any code we wrote should be documentated. It's very important for you to check back and for the people who want to contrubute to it.This is really important in open source community. Because I couldn't find the documentation for the Bookmarks services, I cannot make the bookmark unit testing work. |
Latest revision as of 21:49, 13 December 2006
Contents
- 1 Project Name
- 2 Project Description
- 3 Project Leader(s)
- 4 Project Contributor(s)
- 5 Project Details
- 6 Unit Testing Framework for Firefox
- 7 Project News
Project Name
JavaScript Unit Testing
Project Description
Use an existing unit testing framework (or create one) and demonstrate its use by writing a collection of unit tests that fully test a browser feature of your choosing. The framework will provide decent setup and teardown code that will enable the programmer to start the browser, run the tests, report pass/fail and shutdown the browser.
Project Leader(s)
Paul Yanchun Gu (IRC: gpaul)
Project Contributor(s)
- Vanessa Miranda (vanessa)
- Helped to layout the test window.
- Dave Humphrey (dave or humph)
- Connectted me to Mozilla test team and gave me great links.
- Rob Campbell
- Helped me to detail the requirements of unit testing framework
Project Details
- Collected related topics and examples.
- Learning how to build an extension for fox...
- Had read the links from Dave, it's very interesting, it's called test_nodelist.
October 27, 2006
- Had really hard time on trying to understand how to build and run a test case. This involves the following technologies that I had never learned before:
- XUL
- install.rdf
- chrome.manifest
- Extension Wizard generator
November 9, 2006
- When I try to run the bookmark tester, I got this error:
- Error: uncaught exception: [Exception... "Could not convert JavaScript argument" nsresult: "0x80570009 (NS_ERROR_XPC_BAD_CONVERT_JS)" location: "JS frame :: chrome://bookmarkstester/content/testerWindow.js :: <TOP_LEVEL> :: line 134" data: no]
November 10, 2006
- Fixed exception error, change window.attachEvent("onload", testerWindow.startup(), false) to window.attachEvent("onload", testerWindow.startup, false);
- Trying to get error console working for debugging ...
- dump, setting the preference browser.dom.window.dump.enabled to true in Firefox, then start FF in console: -console
- DOM:window
November 12, 2006
- Try to find most function calls in Firefox, cannot get it work.
- Try to display the firefox console for debugging ,the Firefox 2 cannot brings up console when I tried to user command: firefox.exe -console. In error console from the firefox menu, it displayed an error message: cannot recongnize the command -console.
November 18, 2006
- There two very good references here for MOZ developing
November 19, 2006
- got one bookmark services is working, http://lxr.mozilla.org/mozilla1.8.0/source/browser/components/bookmarks/content/bookmarks.js#432
- but there is no documentation at all in lxr.mozilla.org. when I tried to find to find BMSVC.createBookmark(), I got this BMSVC.createBookmark(aName, aURL, null, null, aCharSet, null), and I don't know what parameter I should use.
- The only think I can do is trying, trying, trying ...
- this is my first unit testing attempt.
- testing addbookmark was working, but it didn't show in the Bookmarks Manager; it will show when I try to search. In google news, one guy had same problem: http://groups.google.com/group/mozilla.dev.tech.xul/tree/browse_frm/month/2006-08/93a6398b0e18ea51?rnum=151&_done=%2Fgroup%2Fmozilla.dev.tech.xul%2Fbrowse_frm%2Fmonth%2F2006-08%3F
This is screen shot before search
This is screen shot after search
- Spent a whole day on trying to debug bookmark services in firefox API... no documentation..., so frustrating.
- ...
- Gave up debugging bookmark services API, continued working on framework.
November 20, 2006
- Coding core code of the framework.
December 06, 2006
- Finished core code of the framework.
- Documenting code.
- Writing instruction on how to write test cases by using this framework
December 07, 2006
- Finished documentation of the framework
December 10, 2006
- Updated the core code of the framework, cleaned debugging code.
- Optimized some functions.
Unit Testing Framework for Firefox
Introduction of this framework
Unit Testing is an approach of programmatic testing of software. The participant will take a section of code and write unit tests for it, so they can find any possible bugs that may exist in the code.
This framework will allow the participant to write unit test cases easily without knowing the UI, messages, and logs. They can focus on write test cases and the framework will take care the rest of the work.
Click on Unit Testing framework extension to download this unit testing framework extension.
Click on Unit Testing framework zip file to download this unit testing framework zip file.
Framework Structure
This framework is an extension of Firefox, it has the common structure that the extension has. This is also a special extension that allows the participant to extend it to many types of unit testing. In following sections, I will use the screen shots to explain the details of the framework.
From the above screen shot, we can see this framework name is unittesting@paulgu.com, and it contains the following folder and files:
- content. It contains the framework code and unit testing code.
- default. This holds default preferences of this extension.
- local. This contains the language settings of this extension.
- skin. This is the style of the extension.
- build.sh, chrome.manifest, config_build.sh, install.rdf, and readme.txt are standard files of the extension. They contains installation information.
This screen shot is showing us a detail list of the famework file in the content folder.
- It contains the jsUnitTest core files which start with jsUnitTest.
- It also contains Unit Testing framework files which start with unittesting.
The files starting with is the core framework files.
- unittesting.js, holds the unit test cases that need to be tested.
- unittestingDisplay.js, is responsible for displaying message to testing window.
- unittestingLog.js, is responsible for logging message to the error console or log file.
- unittestingOverlay.js, is responsible for populating the GUI window.
- unittestingWindow.js, holds all the unit testing functions
- unittestingWindow.xul, is responsible for loading the framework and test window elements.
How to write test cases by using this framework
Writing the test cases by using this framework is really easy. All you need to to is opening the file unittesting.js, and add any type of test case by adding following three setps. (This example is for bookmarks unit testing)
Step 1. Initialize the services
This step initializes the services that contains all the units that are going to be tested. The following is the example of bookmarks services in Firefox API.
/* * test units initialization * * In this part, you need to initialize the services that contains all the units * This is an example for Bookmark Services * ---------------------------------------------------------------------------*/ function BookmarksTests() { logText("initializing bookmarksTests"); this.browserWindow = window.opener; this._RDF = Components.classes["@mozilla.org/rdf/rdf-service;1"].getService(Components.interfaces.nsIRDFService); this._kBMSVCIID = Components.interfaces.nsIBookmarksService; this._BMDS = this._RDF.GetDataSource("rdf:bookmarks"); this._BMSVC = this._BMDS.QueryInterface(this._kBMSVCIID); this.totalTests = countTests(suite); logText("finished initializing bookmarksTests"); } /* * bookmark test window initialization * * initializing services in the firefox API * ---------------------------------------------------------------------------*/ var bookmarkstestWindow = { initialize: function() { logText("Unit Testing Window startup"); this.bookmarksTests = new BookmarksTests(); // initialize bookmark unit tests this.testsRun = new Array(); logText("finished unit testing startup"); } }; /* * initialize bookmarks test window * ---------------------------------------------------------------------------*/ window.addEventListener("load", bookmarkstestWindow.initialize, false);
Step 2. Write a test suite variable called suite that holds all your test cases.
The following is example of bookmarks test suite.
In suite, you could have a setUp() for preparing unit testing, and you could have a tearDown() to clean up after unit testing. The framework will execute them automatically in sequence.
In each test case, you need to run assert() to check the test result.
var suite = { setUp: function() { }, tearDown: function() { }, /* * Unit testing function: testCreateBookmark(). */ testCreateBookmark: function() { var res = bookmarksTests._BMSVC.createBookmark("paulgu.com", "http://www.paulgu.com/", "iso-8859-1", "Paul Yanchun Gu", "", null); // isBookmarkedResource() is not working, // I don't know what type of rSource should be var rSource = "http://www.paulgu.com/"; //check if the bookmark is created successfully var result = true; //assert(bookmarksTests._BMSVC.isBookmarkedResource(rSource)); assert(result, "test Create Bookmark"); }, /* * Unit testing function: testRemoveBookmark(). */ testRemoveBookmark: function() { // need to be developed, this is demo only var result = true; assert(result, "test Remove Bookmark"); }, /* * Unit testing function: testCreateFolder(). */ testCreateFolder: function() { // need to be developed, this is demo only var result = true; assert(result, "test Create Folder"); }, /* * Unit testing function: testDeleteFolder(). */ testDeleteFolder: function() { // need to be developed, this is demo only var result = false; assert(result, "test Delete Folder"); }, /* * Unit testing function: testBookmarkAllTabs(). */ testBookmarkAllTabs: function() { // need to be developed, this is demo only var result = true; assert(result, "test Bookmark All Tabs"); } };
Step 3. Run your test cases
In this example, we run bookmarksTests that we defined in step 1.
/* * run the tests * * This is the main function that trigger the tests * ---------------------------------------------------------------------------*/ function runTests() { runUnitTests(bookmarksTests); }
Customize Extension Menu Item Name
If users want to change the menu item name for a specific test case, they can open file unittesting.dtd under folder local.
And then they can change the first item to your test case name in the file unittesting.dtd.
Run the tests
After adding the test cases to the extension, the participant can install the extension by drag and drop the extension into Firefox, and then click on install to install the unit testing framework with your test cases.
Running test cases is also easy, user can click on Tools menu from Firefox, and you will see an extension called Unit Testing for [...].
Select the Unit Testing for [...], system will pop up the following testing window.
Now click on the run button on test window, the test result will be displayed on test window.
Further more, user can run test cases as many times as they want, and the result will be populated on test window as well.
Unit testing log
Right now, this framework provides a detail log for test running. This will allow users to track the detail testing process and any error that encountered during the testing.
In order to check the error console, go to Tools menu in Firefox, and select Error Console.
After push the Run button, the testing process message will be displayed in console.
Notice: The framework runs the setUp() first, which we can see the highlighting in the screen shot.
Notice: The framework runs the tearDown() when all the test cases are finished, which we can see the highlighting in the screen shot.
What I have done
- Core code and structure of the framework as an extension, and make writing test case as easily as possible. It's similar as writing JUint test cases.
- Testing window layout
- Test result on testing window
- Displaying logs in the error console
- Instruction on how to write test cases by using this framework (readme.txt)
What need to be continued
- Better layout of testing window if you have innovative idea
- Write log into file for further analyzing by Mozilla
- Improve the structure of the framework and make it easier for users if you could
What I have learned
Based on my experice in open source developing, I had a lot of fun for doing it. Also I'd like to address some improtant comments when I was working on the open source.
- It was very time-consuming. Confidence and patience are the key to success.
- This project has no prototype or even any example; most of time I couldn't get help because very less people work on this; and different people will do a different way in unit testing. It took me a lot of time to do researching, trying, and debugging. It also needs a lot of patience; if one way doesn't work, you still need to keep working on it until find the solution. You can take a short break and a deep breath, but never give up.
- Documentation.
- Any code we wrote should be documentated. It's very important for you to check back and for the people who want to contrubute to it.This is really important in open source community. Because I couldn't find the documentation for the Bookmarks services, I cannot make the bookmark unit testing work.
- You need to get help from the others, but don't wait.
- I had sent email to ask help for my problems, and I never get response. It's kind of wasting time. You need to move on, especially in open source community, we do need help from the others, but don't relay on them.
Project News
- Sunday, December 10, 2006 9:45 am, finished core code and structure of Unit Testing Framework for Firefox.
- Wednesday, September 20, 2006 1:07 pm, Rob helped me to get start on JSUnit Testing.
- Monday, September 18, 2006 8:36 am, Dave helped me to connect to Rob Campbell, who is from Mozilla QA team. He explained to me about Unit Testing used in Mozilla.
- Wednesday, October 4, 2006, Saw this interesting post about test cases in Mozilla, and the code example is quite useful.