OSD700 Ref Tester

Goal

Create an easy to implement multi platform framework with the intent to compare webGL canvases and/or webGL buffers with known good or known bad results. Additionally the focus will be on webGL functionality relevant to c3dl, and c3dl rendered canvases.

What is it

The goal of this reference tester is to provide a reliable and extend-able framework to isolate and test webGL functionality in addition to the libraries which use webGL. From the libraries point of view the aim is to isolate what really broke, to point blame on either specific test cases: using small pieces of unique code or isolated internal functionality of webGL. Reasons for a failure are expected to be broken code or a change specifications in the latest implementation.

Plan

Initial steps

Rebuild the framework with modularity in mind. Clearly define and separate the entire process for sanity and easy of use purposes. Areas of focus: test case import, framework setup, specific test case setup, building test case, building reference, generate result of comparison, compile test results. Using that assumption each of the resources being tested will need its on own builder function to generate a testable piece of data. My current idea is to compare all rendered objects using a canvas checking its pixels against a known solution on the other hand I'm unsure how ill approach comparing buffer objects. Additional work will need to be done to confirm how to isolate and test webGL functionality.

Direction

After rewriting the code into a workable framework and developing the methodology of how the tests are being conducted. The work changes pace and focuses more on rounding out the test case base. The focus should primarily be on isolating webGL, particularly the webGL relevant to the c3dl rendering process. In addition to that attention must be paid to breaking down the c3dl rending components and testing them individually.

Tuning

Once a working test base has been established and the framework defined the task becomes fine tuning and optimizing the reference tester. Improving the test case input and organization for scalability eg. after selecting the desired test cases the tester should include directories containing those tests and import them in. This allows the test cases to be separated into files and also be organized by directories. Other necessary improvements would be implementing worker threads to enhance run time of the entire ordeal and especially important the tester should be fully multi platform compatible. Having more platforms under the testers umbrella allows known fails created by build bugs on the part of webGL to because better recognized to the library being tested or Mozilla itself.

Releases

• 0.4

- Clearly identify the test process, and approach

- Rewrite framework defining and separating the testing sequence

- Update readPixels()

- Confirm pixel comparison and blur() actually work

• 0.5

- Implement howto test webGL fundamentals. Both rendered and buffered objects.

- Develop relevant webGL test cases

- Break c3dl tests into more fundamental tests. Isolating very specific objects.

• 0.6

- Port test cases into their own files

- Implement importing test case files

- Define rigid organizational structure, directory and tag based import system.

• 0.7

- Enhance optimization with worker threads

- Identify locations for multiple threads and strategy of use

- Implement multi thread support

• 0.8

- Implement Windows operating system support with all browsers

- Implement relevant webGL test cases for Windows and specific browsers, including known fails.

• 0.9

- Implement Apple operating system support with all browsers

- Implement relevant webGL test cases for Apple and specific browsers, including known fails.

• 1.0

- Implement Linux operating system support with all browsers

- Implement relevant webGL test cases for Linux and specific browser, including known fails.

- Polish code logic: remove unnecessary runtime

Discussion

• How do I:

• Break up the current code?
• Test blur()?
• Include test cases when they are in separate files and directories?
• Approach adding multi thread support?
• Approach moving tests?

• Fixed interval snapshots? How?
• Fixed length, compare result?

• Test specific webGL functionality?
• Test specific c3dl functionality?

• Where do I:

• Find out what webGL is important to c3dl?
• Find OS and browser dependencies?