I just read a very interesting article called Secure String Interpolation in JS that got me thinking about rendering in StringTemplate. I don’t follow the caja project but it looks interesting. I found the article linked from John Resig’s blog.

Script injection in web applications, which also goes by the name XSS (Cross Site Scripting), is a serious and common security problem. It arises when the application does not appropriately escape data that comes from an untrusted source. The data could contain malicious JavaScript that would get executed. If developers are burdened with remembering to escape each and every instance of data they will eventually forget or make a mistake and a security hole will result. One main point of the article is that string interpolation should be automatically secure — it should do the right escaping for you.

Every now and again the issue of how to escape special characters when generating HTML files comes up on the StringTemplate mailing list. The answer is very straight forward - write a renderer. My main point, which I’ll get to eventually, is about the details of the renderer.

The author of the above article distinguishes string interpolation from “full blown” templating languages. JSP, PHP, and XSL were given as examples. Three problems with full blown templating languages were given (folks familiar with StringTemplate know there are more):

1. They do nothing to help solve the escaping problem.
2. They are verbose and add lots of boiler plate.
3. They don’t make simple things simple.

I think StringTemplate holds up well against these complaints. The renders can do escaping. The simple templates (.st files) have no boiler plate at all and the group files (.stg) add only a little. Lastly, it doesn’t get much simpler than “Hello $name$!”.

The most important issue and the one the article is really about is number one — escaping.

On an HTML page what characters need to be escaped depends on the context: element content, attribute value, JavaScript, and URIs. StringTemplate rendering assumes a single rendering context. As of StringTemplate version 3.1 the format option can be used to specify a specific rendering (format). The trouble with this is that the template writer must be aware of the different contexts and what format to use in each. An alternative is to come up with a one size fits all escaping renderer. This will end up escaping more than it needs to. There may be some fringe cases where this could cause problems.

The thing I found most interesting about the approach given in the article is that the context scanner keeps track of the current context in the output so that the escaper can do the right escaping according to the context. To put this in StringTemplate terms the output after rendering would be feed into a parser that tracks changes in context (element to attribute value to JavaScript to element content etc.) and informs the renderer(s) of the current context.

You can still use the format option or other custom renderers to specify how the data is turned into a string (for example a number could be currency or a percentage). This is something that the template author is expected to know. It seems reasonable to me that it would be better if the author didn’t need to understand the intricate escaping rules of web pages.

A key innovation of StringTemplate is the identification of the Model-View-Controller-Renderer pattern — that there is a renderer distinct from the view. (Read about MVCR here.) Now I am seeing the render as two distinct pieces. There is the format renderer and the escaping renderer. The former is concerned with if negative 23 should be output as (23) or -23. The latter is concerned with if single quote should be output as ‘ or ' The format renderer runs first and passes its output to the escaping renderer.

The reason it makes sense to think of two different renderers is that they each have different masters. The format renderer is directed by the needs of the application, the locale and perhaps presentation directed formatting with the format option. The escaping renderer’s function is determined by the language of the output: SQL, HTML, XML or just the text of an email message.

Other thoughts and observations:

Is there a need for a JavaScript implementation of StringTemplate? As it becomes more common to move the model, view and controller to the client there could be a role for it. It could also run server side in Rhino. Although the author of the article states that it is not a goal to create a template language I wonder if the extra capabilities of StringTemplate would be welcome.

The method used in the article for implementing auditable exemptions (cases where you know the data is safe and you don’t want it escaped) is the same as would be done in StringTemplate. Just wrap the data in a type that returns the raw string when rendered.

In the article a simple FSM was used to parse the HTML. Only an excerpt of the code was shown and I didn’t bother to download it and look deeper. It seems to me that the output HTML needs to be well formed otherwise the parser would be very complicated if it has to deal with the tag soup that browsers deal with.

This is now at least the fourth time I’ve seen a parser written in JavaScript so I think there could be a need for a JavaScript backend for ANTLR. I would have liked to create a l-system language parser in JavaScript for a recent project of mine. Is there already an ANTLR for JavaScript that I don’t know about?

Does anyone else think that it would be useful/feasible to have a HTML escaping renderer (that does the correct escaping in automatically) as part of the StringTemplate library?

Recently I had some not so nice things to say about jQueryUI while still professing my love for jQuery. What didn’t come across in that post is that I am hopeful that jQueryUI will become as great as jQuery.

Now it seems I have good reason to be hopeful. John Resig tells us the good news that Paul Bakaus (the lead developer) now works full time on jQueryUI at Liferay and that a new release is due out soon.

While playing with Scratch I was thinking about what kinds of things would make good Scratch projects. I have had an interest in L-systems for a long time now but never did more than read about them and look at the pretty pictures. I thought scratch might motivate me to do something in this area but I quickly realized with Scratch’s lack of data structures (besides the 2d drawing surface) and argument passing it would take all the fun out of doing an L-system implementation. Sure, people have come up with ways of drawing fractals in Scratch but these are fixed algorithms usually at a specific generation. I have not yet seen an L-system implementation in Scratch.

About a month later I went to the ajax experience conference and learned about the new HTML canvas element among other things. I wanted a project to get practice writing JavaScript, more experience with using jQuery, and to try out jQueryUI and canvas. L-systems seemed like a great project for this purpose.

Before I get to my project here is a real quick definition of what an L-System is. A Lindenmayer system (L-system for short), named after biologist Aristid Lindenmeyer, is a rewriting system. It takes a list of symbols and repeatedly generates a new list of symbols by following a set of rules. Each new list of symbols is called a generation. The symbols can be given a graphical representation to produce an image algorithmically. The beauty of L-systems is that with just a few symbols and rules amazingly complex images can be created. L-systems were developed to model plants but can also draw fractals. More information is available from the links above and from the lsystem.js documentation.

My project, L-system.html, is a serverless web application for exploring a number of plant and fractal models created with L-systems. You select a model, change its parameters such as angle or distance depending on the model, set the number of generations then press the play button to see it drawn.

Being able to adjust the parameters of a model helps to understand it. When you see a fractal of some reasonably high number of generations it is not always clear what process gives rise to it. Stepping through from the zeroth to first and second generation of a fractal really helps to visualize the rule that makes the fractal what it is.

L-system.html is not very fancy or even complete. The main purpose for me was learning canvas, jQuery, and jQueryUI. It makes use of two reusable JavaScript libraries that I wrote:

• lsystem.js – a L-system library that supports parametric or non-parametric, deterministic or non-deterministic, context sensitive or context free, and branching or flat models.
• turtle.js – A 2d turtle graphics library on top of the HTML canvas API.

Both of these are open source with a BSD license if you care to use them. They can be used together or independently. You can download the whole project. If you make improvements to them or have suggestions I’d love to hear about it.

The rest of this post covers what I discovered along the way.

First I have to say I’m impressed with canvas and its capabilities. Its not as full featured as most mature graphics libraries and there are a few features that are not supported on all browsers but it opens up another area to web applications (without having to resort to Flash) that was previously only found in desktop applications. People are doing cool things with it already. Support for IE is accomplished with explorercanvas from Google. I have tested my project on Firefox 2, IE 7, Opera 9.24 and Safari 3beta (on Windows) and it works well except that the save functionality doesn’t work in Safari or IE. Also I have noticed IE to be slower than the others but have not done any rigorous timing.

Thanks to Canvas Paint (a canvas implementation of Microsoft Paint) for the technique for saving canvas images.

L-system.html is an example of a serverless web application. Its an application because it is interactive and does something at least a little interesting or useful. Its a web application because it runs in a web browser. Serverless is a somewhat new category of web application. It basically means that none of the application behavior is running on a server. A web server may be serving the HTML and related resources but there is no servlet, or PHP script, etc. A server may be providing raw data or persisting data (like the Amazon S3 service) but this is just data storage or acquisition and often the servers or services are not even owned by the creator of the application. In some cases the application can just be an HTML file on your local file system. L-system.html can run this way. An even cooler example of the kind of interesting things that can be done without a server is TiddlyWiki - a serverless wiki.

After working more with jQuery I’m convinced that this is the way I want to write JavaScript and to work with the DOM. However I’m not anywhere near as impressed with jQueryUI. It just doesn’t seem ready for prime time. I only wanted a few UI elements: a slider, a dialog, a progress control and a color picker. There is no progress control or color picker in jQueryUI although there are plugins for color pickers and maybe a progress control as well. I thought it would be best to stick to the official UI widgets. I struggled to learn the UI widgets and get them to work the way I wanted. I needed to read the source as well as the documentation to figure them out.

The first problem I ran into was with the slider on on IE 6 and 7. I found a bug report that said the problem was fixed in the latest. After downloading the latest from the SVN source repository the problem was indeed fixed.

I wanted to be able to move the slider with the right and left arrow keys and to show that the slider had focus. I know that jQueryUI doesn’t have full accessibility support yet, but keyboard support seemed like it should work for a control like slider. I noticed some code to turn on keyboard support when webforms (not sure what that is) is present. This helped me figure out how to get it working the way I wanted. You can tab to or click on the slider to give it focus and then use the arrow keys to move the slider.

One thing that bugs me about some ajax apps is that they break the back button. I really wanted to get back button history working with jQuery in this project. At the ajax experience conference I saw the history plugin demonstrated and it looked so easy. So far my attempts to get it working have failed. The history plugin seems like it would be easy to use with the tab control but i’m not using the tab control. Then I read that the new tab control in jQueryUI doesn’t work with the history plugin. There is a mention that the history plugin needs to be rewritten but no time frame for when it will be available. Since I’m not using tabs it may be possible to get the current history plugin working but I haven’t figured it out yet.

My next problem was with the dialog control. I found that I could end up with multiple copies of the dialog displayed. I decided to try the latest version from SVN like I did with the slider. I did manage to solve my problem by using the new dialogInit method. (There may have been another way.) The trouble is that at the time I got the latest version of dialog the mouse management was rewritten which broke the slider and resizing of dialogs. As of this writing they still are not fixed. There may be some set of source versions that will solve both my slider and dialog problems but I don’t have time to find it. So at this point the sliders do not move with the mouse (you get a JavaScript error). I’ll just have to wait for the jQueryUI mouse implementation to get finished.

I noticed that some of the jQueryUI documentation (at least the examples) are in advance of the current 1.0 release. For example, the dialogInit method was shown in one of the dialog examples even thought it is not in the 1.0 release.

There is no progress control in the jQueryUI so I made my own. The next step will be to learn how to turn it into a plugin.

Along the way I found a number of useful tools and practices for writing JavaScript

• Firebug is a very good tool. I knew that already but with this project I used the profiler for the first time. The profiler is very easy to use and helped me to improve the performance of the L-system algorithm. I also used the console log, which Firebug lite makes work cross browser.
• I’m using JSLint from an ant task and it has found some problems. I described the technique for ant integration here.
• I used jsdoc-toolkit to generate the library documentation. This tool is very easy to use and produces nice results. Since the JavaScript is going to get minified anyway I think it is a good idea to keep the documentation in the source.
• For testing the 2 library modules I used jsunit. Actually I only used the assertion functions from the core. This looks like it could be a good tool but I need to learn more about how to use it effectively.
• I use just the CSS library from YUI.
• To build the final html as well as the aggregated and minified resources I use ant, yui compressor, and STST. One thing I noticed is that yui compressor catches some JavaScript problems that JSLint did not.

After I was just about done with this project I discovered that someone else created a canvas L-system implementation LSys/JS. Although it doesn’t support as many kinds of L-systems it does allow you to enter your own. Also it does not have a reusable library layer.

If you have read this far without trying it out you should take a look at L-system.html now. If you want to add your own L-system models you can download the project and modify the lsystem.html file. If you come up with a cool model leave a comment with a link to it.

I just attended the 2007 Boston Ajax Experience. Its been a while since I attended a conference like this. It is nice to have an employer (BEA) that recognizes the value in such things. The conference was excellent. It was full of great topics and the presenters were top notch.

I’m not going to describe each session but I’ll mention a few that stand out in my mind. I’ll also throw out some random thoughts, impressions and lessons learned.
Continue reading…