Processingjs paper

Js and processing integration

Processing is Java based, and in order to make it work in the web, it has to be completely converted into JavaScript. Syntactically JavaScript and Java are actually quite similar, and people have done work like this before (google, java nes emulator to js nes emulator). Our unique challenges were that we had to do this dynamically, be fully object oriented, support all native Java functions that are supported by Processing, and consider all web like differences, like images having to be pre loaded before we can start processing the code, casting typeless variables, function overloading, and variable name overloading.

We could of done a straight up JavaScript port of the Processing language, but that would mean all Processing sketches written in Processing, would need to be rewritten in JavaScript. This way, all previous Processing sketches can simply be dropped into the web, and they will work. We took this one step further, allowing both languages to mingle as one. When we parse the Java into JavaScript, we don't break previously existing JavaScript, this means you can add JavaScript right into the Java, without having to declare that you are doing so. We simply ignore the JavaScript we encounter while parsing the Java, leaving it in tact. Not only do we allow mingling of the two languages, which is unique and powerful in itself, but also allows for sketches to be written in pure JavaScript. The advantages of this is we had a huge library of work to test and draw from right from the beginning.

John Resig, the mastermind behind Jquery, is also the mastermind behind Processing.js. His initial work was to use regex to scan the sketch source code for hints of Java, replace it with JavaScript, and leave all JavaScript in tact. He started by taking a previously existing Processing sketch, adding functional support to make that one sketch work, and doing this one sketch at a time, creating missing functions as needed. He took advantage of the pre existing library of sketches, so for each sketch he explicitly supported, he would be that much closer to implicitly supporting other sketches.

“In development I worked in a backwards manner. Instead of building the API up from the ground - I worked from the top, down, implementing enough of the API to get individual demos working.” -http://ejohn.org/blog/processingjs/

Scott Downe's work was mostly related to fixing bugs, and removing the dangerous JavaScript function with. Fixing bugs was a good place to start learning the code, getting his feet wet. The first bug he fixed was to make sure potential code contained in strings were not parsed. This was initially accomplished by masking all strings with a key, and storing their values before the code was parsed, and later replacing the unchanged strings via their keys after parsing. Other, smaller bugs were fixed until it became apparent that the use of the with function meant we would fall off trace, and wouldn't reach our full speed potential. With was being used in two places, first being around all of the sketch, to load in the whole of the Processing library, and to load in method calls from internal function use. We have to do this, because of the differences in how Java and JavaScript call and access their object properties. JavaScript accesses all properties within the object itself separated with a dot from inside or outside the object, where as Java only needs a dot when accessed from outside the object. Using with meant we could contain all Processing functions inside an object, and not have to change how it is called inside the Java. This was the easiest and fastest way to do this, but needed to be changed. Removing with meant prepending the processing object to all calls to the API and internal object properties. So we needed to store a list of the existing properties for both the API and created objects, and when the parser finds a match, prepends itself, either being “Propcessing” or “this” to the property. This worked, but was fragile; we were still using regex's, and doing this to the whole of the source, meaning each new regex we called was a danger to parse code that is similar, but different, potentially breaking code we did not intend to that previously worked. Despite working, this was a hack and a maintenance nightmare. We needed something better.

Notmasteryet rewrote the parser to convert the sketch into an abstract syntax tree, which is an abstract tree representation of blocks of code. By doing this, blocks can be precisely parsed without the worry of breaking or parsing unintended things in an unexpected way. Regex is still used for each part, but is now contained to specifically targeted smaller chunks code, instead of the whole thing. This makes maintaining the code much easier, makes object inheritance easier, and makes JavaScript code included in the sketch more stable. In fact, since the abstract syntax tree's inclusion, we have found new bugs in the parser to be pretty much non existent.

Each of the above people contributed object inheritance in some form or another, but I wanted to specifically touch on the challenges in inheritance. Object inheritance was much easier using with, because we could easily add the inherited properties to an object, and when called, not worry about where it is being called from. When with is removed, we had to maintain this data internally, and be able to prepend the right object to the right method calls. This got significantly more complicated when you consider where things may be called from, including super constructors, and super methods calling methods form its parent, calling these potentially chaining calls in the correct order. Because we have to store all created classes methods at the time of parsing, we don't yet know if another class will use it as a super class, so all classes and their properties must be stored, so later we can prepend the correct object to the correct calls in a complex chain of limitless inherited calls. This was buggy and fragile code that took a while to get right, but Notmasteryet's work helped a ton in this area, and something we are quite proud of.

Some of the differences between Java and JavaScript presented some unique challenges. Some of which are still unsolved. Because at the time of parsing, we are just parsing the code as if it was pure text, so we cannot validate any of the data referenced in the code. When an image is to be loaded in the code, the client will now have to download that image from the server, this is a unique problem that Processing does not have. This means an image may not be available when needed, and getting that data directly from the source at time of parse is not reliable, we would need to know this before we parse. We solved this by adding a directive at the top of the code that would define all images needed to be preloaded, so we can parse the directive first, then convert the code to JavaScript, then run it, safely knowing images will be ready to use at run time. Java supports overloading, in that its functions are uniquly identified by their name, return type, and parameters, this making up a function's signature. ( - source this ) JavaScript only holds the function name as its signature, presenting another unique problem. We can check the number of parameters in a function, and merge all overloaded functions into one, and check the number of arguments passed in, to know which block to call. This check is at run time, not at call time as Java would do it. However, we currently do not reliably check the type of the arguments passed in, so it will break if a function has two versions, first accepting a single string as the only argument, and the second accepting a single number as the only argument. Similarly, if we have a variable using the same name as a function, called variable name overloading, we will break in the same way. This is because Java would consider these different things, and JavaScript considers a function to be a variable of a different type, sharing the same space.

“In order to support this there would have to be considerable overhead - and it's generally not a good practice to begin with.” -http://ejohn.org/blog/processingjs/

Another interesting difference stems from Java being a typed language, and JavaScript being typeless. Java would require casting in most cases, where as with javaScript we can simply throw the cast away for all literal variable types. The problem is if the type is something like a double, or a char, which in JavaScript is simply a string or int. ( source this? ) We solved this for chars with a custom char class, it solved a lot of issues we were having but it is not perfect, by not solving all issues in all cases. Some other types like double and byte will require more overhead and will not be possible without complete type tracking.