9
edits
Changes
no edit summary
=Processing.js Paper=
=Processing.js Paper=
==Introduction==
Data visualization in a webpage beyond images typically required some sort of plug-in. However due to security concerns and general wariness to plugins, they are not the most effective means to deliver content. Furthermore there are often some platform where a plugin does not exist or cannot exist. Even Flash which is one of the most ubiquitous visual environments are not available on every platform. The only real solution to web delivery of rich graphics is to integrate it into native browser technology.
The HTML <canvas> element allows the programatic delivery of graphics in a web page without plugins. With its inclusion in the soon to be released IE 9, the <canvas> element now represents a means to deliver graphical content in all the major browsers. The typical way to draw within a canvas is to use javascript but for artists, educators, and other people less familiar with Javascript, learning to do this can be a barrier to entry.
The Processing language introduced by Ben Fry and Casey Reas is a simple and elegent language for data visualization that is already used by artists, educators as well as commercial media to deliver rich graphical content called sketches. There is a large body of work around the world which had been previously developed using Processing. However, this is largely not something that is consistently delivered through a web page. This is due to the fact that Processing was originally developed with Java and thus delivering Processing sketches required that the user install a Java plugin. Furthermore the sketches themselves were self contained items as opposed to being part of a web page. That is, the elements of the Document Object Model (DOM) of a webpage could not interact with it or vice versa. Thus, while it was possible to deliver visual content it would be difficult to create Processing sketches to take full advantage of modern web services such as flickr, twitter etc.
Processing.js is an open source, cross browser Javascript port of the Processing language. It uses the canvas element for rendering and does not require any plug-ins. However, Processing.js is more than just a Processing parser written in JavaScript. It also enables the embedding of other web technologies into Processing sketches. This extension will allow for a new set of visualizations previously not possible. Processing.js seamlessly integrates web technologies with the processing language to provide an accessible framework for multimedia web applications.
==Background==
The processing.js project was started by John Resig who wanted to utilize the HTML5 canvas element and take advantage of the Java Processing language. It took about seven months to get a working version, consisting of 5000 lines of code but it was not a complete port of the Processing language. (Resig 2008).
“In development I worked in a backwards manner. Instead of building the API up from the ground - I worked from the top, down, implementing enough of the API to get individual demos working.” -http://ejohn.org/blog/processingjs/
The project, similarly to other open source products, was released with the hope that a developer community will converge around it and contribute to development.
"The Mozilla experience however, suggests that proprietary products may not be well-suited to distributed development if they have tightly-coupled architectures. There is a need to create an “architecture for participation,” one that promotes ease of understanding by limiting module size, and ease of contribution " - (MacCormack, Rusnak and Baldwin 2004).
In September 2009, we began the work to complete the Processing port to JavaScript. In order to facilitate an architecture for participation a number of things needed to happen. First and foremost the source code had to be readily available. Secondly, the inner workings of the project and the missing functionality must be publicized and a dialog started. To this end the source code was made available publicly on GitHub and an issue tracking system was used to manage the large number of issues needed to be resolved in order to complete the port. A review process was setup to ensure that the code submitted was of sufficient quality.
==DOM Integration?? (need a better header)==
Processing.js is more than just a Processing parser written in JavaScript. It is designed in a way that connects the Processing language (also known as P5) with web technologies such as JavaScript, the HTML5 canvas element, JQuery, and various web services. Furthermore, Processing.js is built in such a way as to allow easy integration of new technologies as they emerge.
The original Processing Language is Java based. To run a Processing sketch in a web page, the Java code has to be completely converted into JavaScript. While syntactically JavaScript and Java are fairly similar, there are some fundamental differences that has made this conversion challenging. The first is that we wanted to do this conversion dynamically in real time. The code produced by the converter needed to be fully object oriented and we had to provide support to all native Java functions and objects (such as Strings) that are supported by Processing. We also had to take into account the differences between working with web resources vs local resources. Furthermore we had to consider how we would handle some fundamental differences between Java and JavaScript such as typed vs. typeless variables, function overloading and variable name overloading.
From it's inception, Processing.js was designed to be more than just a rewrite of the Java functions provided by Processing to JavaScript. John Resig wrote the original Processing.js parser to scan a Processing sketch for hints of Java code and convert that code to JavaScript. However, if the parser encountered JavaScript code, it would leave the code intact. This method allowed not only for the conversion of existing Processing code to JavaScript but the injection of JavaScript into Processing sketches as well. This simple idea means that within a processing sketch Java and JavaScript code can exist together without any need to declare the language you are using.
==3D support==
The introduction of the <canvas> tag into the HTML5 specification allowed Processing to be ported to JavaScript, thus enabling users to run 2D sketches within the browser without additional plug-ins. At the time when porting began, there was no plug-in free method of delivering 3D content. This limited Processing.js to its 2D functions. WebGL, A JavaScript API that is based on OpenGL ES 2.0, is now being implemented by Firefox, Chrome and Safari. It is now a viable candidate for use in Processing.js to render 3D sketches. Additionally, since WebGL closely matches OpenGL which is used by Processing, it substantially aided the porting process.
===Differences===
The matter of porting Processing (which uses OpenGL) was simplified because the WebGL interface is similar that of OpenGL, but there are a number of differences between the interfaces. Arguably, the single largest difference between WebGL and OpenGL is that like OpenGL ES 2.0, the fixed-function pipeline was been removed. Because of this, not all Processing source code could not be ported directly. Instead, user-defined vertex and fragment shaders were necessary to write for lighting operations. Since some shapes in Processing aren't lit and others were, multiple shaders were written. One shader exists for lit objects such as boxes and spheres, another less complex shader was written for unlit objects such as lines and points.
The following shaders are used for rendering unlit shapes specified with begin/end function calls.
<pre>
"varying vec4 vFrontColor;" +
"attribute vec3 aVertex;" +
"attribute vec4 aColor;" +
"uniform mat4 uView;" +
"uniform mat4 uProjection;" +
"void main(void) {" +
" frontColor = aColor;" +
" gl_Position = uProjection * uView * vec4(aVertex, 1.0);" +
"}";
</pre>
fragment shader:
<pre>
ifdef"GLfESf GL_ES\n" +
"prehighpn highp float;\n" endif"#endif\n" +
"vvecinvFrontColorntColor;" +
"void main(void){" +glrFragColoragCvFrontColorntColor;" +
"}";
</pre>
Examining the shaders reveals some of the idiosyncrasWebGLf WebGgl The gl_Color keyword is considered invalid. Instead, users must create their own varying vector. Furthermore, a preprocessor statement to set float types to use high precision is also required. These are some examples of changes to the specifications changes which were introduced over time.
===Typed Arrays===
Performance is always a concern when rendering 3D content, so it was necessary to create a faster version of JavaScript'script's inherently slow arrays types. Because of this, typed arrays were incorporated into pre-release versions of WebGL browsers. Unlike regular arrays which can contain different types such as strings, numbers and objects, typed arrays can only contain one type and cannot by dynamically resized. Some of these types include Float32Intay, Int32Uinty, Uint16ArrUintnd Uint8Array. These types provide a significant performance increase when manipulating arrays.
(table removed)
<table border="1">
<tr>
<td>Operation</td>
<td>Array</td>
<td>Float32Array</td>
</tr>
<tr>
<td>Write</td>
<td>8947</td>
<td>1455</td>
</tr>
<tr>
<td>Read</td>
<td>1948</td>
<td>1109</td>
</tr>
<tr>
<td>Loop-copy</td>
<td>>10, 000</td>
<td>1969</td>
</tr>
<tr>
<td>Slice-Copy</td>
<td>1125</td>
<td>503</td>
</tr>
</table>
Win7 64Bit, 4GB Ram, Dual-Core 1.30Ghz Intel U7300
(citation needed)
Alistair MacDonald
[http://weblog.bocoup.com/javascript-typed-arrays link]
Because typed arrays are only available for pre-release browsers, they cannot currently be used in 2D sketches. Once they become implemented in browsers, a significant amount of the Processing.js code base can make use of these structures, increasing performance throughout the library.
===Specification Changes and Browser Inconsistencies===
As the specification is concurrently implemented in different browsers, several inconsistencies between browsers have appeared. These range from minor issues, such as Minefield and Chrome/Chromium return "function" while WebKit returns "object" when the type of a typed array is queried. Another is the way WebGL's readPixels() function is implemented. This function isn't used extensively in the library itself, but it is used in the Processing.js reference testing framework.
===Problems===
WebGL provides a close match to OpenGL for incorporating 3D into Processing.js, but it does present some issues when trying to port over code. There are interface differences, changes to the interface are common, and some functionality isn't available at all such as point smoothing.
==Browser Unification==
One important feature provided by Processing.js is that it hides the differences between browsers. Web standards are often loosely defined, and thus variations can exist. These variations not only exist between different browser vendors but can even exist between versions of the same browser on different platforms. Something as simple as key events can vary widely between browsers. Processing.js hides all these intricacies from the user keeping it simple for content creators.
=Notes=
==Introduction==