Category Archives: JavaScript

Using Oracle JET from APEX

This post contains statements about possible future functionality. These are not official statements. They are speculative, subject to change, and should not be relied upon.

Reading svenweller’s recent article Integrate Oracle JET into Apex 5.0 got me thinking about this topic. I’m very busy working on Interactive Grid in APEX 5.1 right now but this is an important topic and there is a need to get some information out sooner rather than later. I’m not going to go into how JET and APEX are great, why it makes sense to use them together or provide step-by-step instructions for doing so. See svenweller’s blog for that.

By now most APEX developers have heard that we plan to include Oracle JET based charts in APEX 5.1. It is generally easy to use 3rd party JavaScript libraries with APEX. Here I want to talk about why Oracle JET is a special case and give some tips for making them work together. I want to share what we have learned so far in integrating JET charts with APEX. I should mention that I am not the main person working on the chart feature.

What makes using Oracle JET with APEX tricky is that JET requires you to use RequireJS. So the same issues apply if you were to integrate any code that uses RequireJS with APEX. It is not specific to Oracle JET. Furthermore, you would have similar issues trying to use JET with any traditional web app that isn’t already using RequireJS.

Is RequireJS bad? No. Is APEX somehow at fault? No. What’s going on here? Its complicated…

The traditional way of using JavaScript on a web page is to put it in script tags and call it. Lots of libraries? No problem. Just figure out what order to put all the script tags in. Each script file adds some global symbols for the others to use. Namespace objects are used to reduce the number of globals — its a kind of modularity. This is what APEX does. It is conceptually simple.

Then modules were invented as a way to keep code, well, modular and explicitly define what your module exports and what dependencies you have on other modules. Just one script tag and the module system (in this case RequireJS) does all the loading. The only globals are from the module system (define, require, requirejs). This is also conceptually simple but different concepts.

There is nothing wrong with either of these ways of building web apps. I have nothing against modules. I like using the Node.js module system. But APEX has a long history and simply cannot switch over to using RequireJS modules without breaking existing applications.

These are two very different worlds that don’t mix well. Its hard to ease into using modules because of what happens when the existing non-module code and the new module code wants to share a library.

If you are the creator of a handy library you want to increase your user base by being usable in both worlds. (I’ll use Hammer.js as the example simply because it is a library that both JET and APEX Universal Theme use.) Its easy to do, just add at the bottom of your code something like the following:

// this is just slightly simplified from the actual hammer.js code
if (typeof define == "function" && define.amd) {
    define(function() {
        return Hammer;
    });
} else {
    window.Hammer = Hammer;
}

Now the library can be used in a module system if the page is using one or just define a global for the traditional way.

But if a single page has some code that uses RequireJS and some code that uses the traditional way and both want to use the same library (Hammer.js in our case) there is a problem.

A) If the hammer.js script tag is loaded after require.js it defines an anonymous module as shown above and then as soon as you do anything else with RequireJS you get the error: Mismatched anonymous define() module. The reason is that RequireJS insists that all anonymous modules are loaded from a define or require call. The assumption is that as soon as RequireJS is on the page you are using the new modular way.

B) If the hammer.js script tag is loaded before require.js it defines its global Hammer object so that traditional code can use it. Then when module code uses require(['hammer'...]...) or something equivalent it will get a new copy of the Hammer object. You can prove this with the following code:

require(["hammerjs"], function( h ) { console.log( h === window.Hammer );});

If the the new module code and the global Hammer object were the same it would print true but it prints false. At least there is no error and it seems to work but… Ideally we would like to avoid this. Depending on what the library does it could result in more than just wasted memory.

We had a similar problem with jQuery where after RequireJS was used by JET Charts all our APEX jQuery UI widgets were essentially “gone” because there were two different jQuery objects ( $ !== apex.jQuery ). Note jQuery is a special case in that it defines a named, not anonymous, module so it doesn’t suffer from the error in case (A). You may also say that it is best to always use apex.jQuery rather than $ and you are correct but not everyone follows this rule all the time. And we still don’t want to waste memory on duplicate copies of libraries.

With this background out of the way here are my recommendations.

  • Load require.js as late as possible because problem B is much easier to deal with than problem A. (How to deal with problem B is coming up.)

    RequireJS would like to be the first (and only) script tag on the page but it doesn’t have to be. In order to keep any RequireJS aware libraries such as hammer.js or jQuery from calling define, require.js should be included after them. The best way to do this is to add require.js to a page in page attribute JavaScript: File URLs.

    With this approach you don’t need to throw out hammer.js just to use Oracle JET. Hammer.js is useful when running your app on touch devices.

  • You don’t need to use data-main on the require.js script tag. It is not required, just a convenience. Your code that includes the call to requirejs.config can be handled like any other JavaScript code. If you put it in a file, reference that file in JavaScript: File URLs (after require.js). It can even go inline by putting it in JavaScript: Execute when Page Loads.
  • To get around the problem of loading multiple copies of a library add the following code after the call to requirejs.config and before any calls to require
    define( 'hammerjs', [], function() {
        return Hammer;
    });
    define( 'jquery', [], function() {
        return apex.jQuery;
    });
    

    What this does is define the library module with a name in the RequireJS module system so that it will not be loaded again. It is important to use the same name that the library uses in the config paths object passed to requirejs.config. This can be extended to other libraries if needed.

    Note: If different versions of a library are needed then you may just have to let two copies be loaded. In this case remove the corresponding call to define.

  • You probably don’t need to use knockout. I don’t know for sure if this is true for all JET components but it is true for JET Charts. APEX 5.1 is not using knockout at all. I have nothing against knockout. However, it is of most benefit if it is used pervasively through out an app. Otherwise it is just another another library to load for which there are other ways to do what it does. That said, if you want to use it go ahead.

At this point I don’t know what if anything APEX 5.1 will do to make integrating Oracle JET (or RequireJS in general) easier. For now I hope this helps anyone trying to use JET with APEX 5.0.

StringTemplate for JavaScript

About three weeks ago I announced on the StringTemplate discussion forum that I had updated my StringTemplate Standalone Tool (STST) to work with StringTemplate version 4. It has been a number of years since I have used StringTempalte and almost as many since I’ve programmed in Java. So why this return to StringTemplate and Java? My motivation for updating STST is that I intend to create a pure JavaScript implementation of StringTemplate.

So far my work with JavaScript has mostly been creating UI widgets as part of existing web apps and frameworks. I have not had a need for templates, at least not ones processed by JavaScript. That may change with my Climbing Comp project and some other code generation projects I have in mind.

What I’m looking for in a JavaScript template engine:

  • Strong separation of presentation and logic
  • Compiled templates, good performance
  • Mostly need to run on the server using node.js but running in the browser is a plus
  • Can generate any text format, not just HTML

The best match I found is Handlebars. It compiles to JavaScript, runs in the browser and node.js environments, and does a decent job of keeping logic out of the templates. It seems focused on HTML generation but I think it could work for other text formats. I have only played with it a little. There are a few things that I don’t like. Having to write helpers in JavaScript and the way partials are handled distinct from templates doesn’t seem as nice as StringTemplate. Handlebars doesn’t have the separate renderer and writer layers that StringTemplate has. I didn’t do a detailed analysis of the differences between Handlebars and StringTemplate. My gut tells me that I would prefer StringTemplate.

Because, at least initially, I plan to do generation off-line on the server side I could use the Java version of StringTemplate but I would prefer not to add Java to the tool chain; so far I have been able to do everything I need with node. This is what has brought me to creating StringTemplate for JavaScript.

StringTemplate for JavaScript (StringTemplatet-js) will compile templates to JavaScript. It will not have an interpreter. It should be 100% compatible with StringTemplate v4 template syntax and semantics. The API will be different.

To implement StringTemplate-js will require parsing template files and generating JavaScript code. The former would be a good job for ANTLR and the later well suited to StringTemplate. ANTLR can generate JavaScript code but again I would rather not have to use Java. This lead me to try two JavaScript parser generators: jison and pegjs.

I tried jison first and it was a failure. Any syntax error in the grammar resulted in a generic exception with no indication of what went wrong or even the line number. I found my errors by selectively commenting out code until the problem line was isolated. The generated code was extremely difficult to understand and debug. Then when I tried examples right out of the documentation that didn’t work I gave up.

Now I’m using pegjs and it is working very well. The error reporting is very good and the parser it generates also has very good error reporting including source line and column. One minor issue is that it stops at the first error. The generated code is not as readable as what ANTLR3 generates but it is reasonable and I have had no problem debugging it. Not having a separate lexer and parser takes some getting used to. It means having to represent white space throughout the grammar because you can’t just toss the tokens out in the lexer. Some of the things I thought would be difficult, such as having configurable start and stop template delimiters, turned out to be quite easy to deal with.

StringTemplate-js will use itself to generate the compiled template JavaScript code. This is where STST comes in. I plan to use it to bootstrap the process. It will also come in handy in validating that StringTemplate-js gets the same answer as the Java reference implementation.

Hopefully soon JavaScript will have another template engine option. Watch this space.

Scoring Rock Climbing Competitions

The last two years or so I have been very busy working on APEX 5.0. This is the main reason for not posting anything in so long. It’s not that I was so busy with work that I did nothing else, but just that I didn’t have the motivation to write. I should have plenty to write about APEX soon. But this post is about another interest of mine; rock climbing.

My youngest daughter climbs competitively and I try to climb at least twice a week when I bring her to team practice. The local climbing comps consist of proud parents watching their kids climb the hardest routes they can and then, when the scorecards are turned in, waiting around for an hour or more while the scores are tallied. It is not uncommon for people to leave, get lunch, and return before results are announced. In our area, climbing comps typically have 100 or so competitors.

In early 2014 a climbing friend, Matt, who was also responsible for scoring an upcoming competition had some ideas for how to make the scoring go faster. He asked me what it would take to create an app to do scorecard data entry. I saw the project as an opportunity to learn some new technologies I was interested in, specifically node.js, and jQuery Mobile. I picked a mobile web app because it is the best way to reach the widest range of devices with the least effort.

Climbing comps are traditionally scored using an Excel spreadsheet. For each scorecard you look up the row of the climber by name and then enter the number of points for the top 5 climbs and the number of falls (or attempts). The spreadsheet creates a bottleneck because only one person can be doing the data entry. Even if the bottleneck were removed by splitting up the spreadsheet by gender and/or category (or using Google Sheets), the data entry would still be tedious and error prone because of all the numbers that must be typed in. This is why it takes so long to get the results and give out the ribbons so the kids and families can go home and the climbing gym can open for regular business.

I worked a few weekends on the climbing comp mobile web app. I already enjoyed programming in JavaScript and learning node.js was a lot of fun and pretty easy. I chose a thin-server architecture with MariaDB for the database, restify on node.js for the REST resources and static content, and jQuery Mobile for the front end. I specifically wanted to minimize the number of libraries and avoid using a heavy weight MVC framework. This was to reduce the number of new things to learn at once, focus on the things I did want to learn without additional layers, and experience the pain points first hand, so I could better choose what additional libraries or frameworks I might want to add or switch to in the future.

By the time of the first competition, the app was working well enough to be used. The database had to be loaded with climber and route data with command line scripts, but the UI for data entry and reporting results worked. The key point of the app is that the UI closely resembles the score card and it already knows the point value of each route. Data entry is simply a matter of entering the 3 digit climber number from the score card using an on screen keypad, touching each route they topped, and touching the number of falls; 7 taps. The UI is optimized for tablets, but works well on laptops as well, and the UI is responsive so it can even be used on a phone.

On March 15th 2014 we used the climbing comp app for the first time at the Boston Rock Gym. The server and database were running on Matt’s laptop. We had between 3 and 4 people doing data entry on iPad or Android tablets. As scorecards came in, they were divided between the people doing data entry and quickly entered into the app. When time is called, any climbers that are waiting in line get to finish their last climb. By the time the last climber was on the wall, all the other scores had been entered. The owner of the gym had the honor of entering the last scorecard and the scoring was complete before the last climber’s feet touched the ground. The app was a great success. On the down side at least one family, expecting a long wait, left for lunch and returned to find that the ribbons had already been given out.

Since then, the app has been used successfully at two more comps. Before each one I have added new features and fixed bugs. Now climber and route data can be entered or uploaded from the app UI rather than using command line scripts. There is plenty more work to be done but it is just about at a point where others could use it. (Currently, it only works for red point format comps.) ClimbingComp is available on github for anyone to download and use.

Angular, Bindings and Templates

I recently watched a video about a cool new technology called <angular/>. It looks like a great way to create web applications. It caught my attention because I saw in it some parallels with some of my recent writings and what I am currently working on.

The video was very good but in the beginning Misko Hevery struggled a bit to describe it and at the end admitted the biggest problem was explaining to people what <angular/> is. To me it is clearly a language, and an extensible one at that. It is (or at least contains) as he describes an HTML compiler. Doesn’t a compiler imply some language to compile. The source is clearly more than just HTML so it must be more than just an HTML compiler. I wonder if there is some reason he avoids calling it a language. The fact that it can be extended to handle new behaviors, widgets, etc. just makes it a better language. Is there some fear that calling it a language would put people off? It doesn’t bother me since I previously said that higher level languages will provide the biggest productivity gains.

The parallels I was talking about are:

  • Its a declarative language
  • The language is designed to be extended
  • It has two way data binding

Two way data binding is a very important aspect in simplifying the creation of web apps. The first time I ever saw data binding was back in the the late ’80s. I was visiting a friend that was working on a maser, which is cool (literally) in its own right, but what caught my eye, as he showed me around his lab, was some software running on an old Mac. That software was LabView. The fact that a line drawn between a sensor and a gauge meant that any time the senor value changed the gauge was updated really impressed me. More recently I noticed that data binding is built in to the JavaFX language. I haven’t written any programs in LabView or JavaFX so I’m not sure if the bindings can be bidirectional but I’m pretty sure at least in JavaFX that bidirectional changes can be accomplished.

It is the two way data bindings that most notably distinguish <angular/> from template languages. I used to be a big fan of template languages, and StringTemplate was my favorite. Lately I have found them to be less useful. One reason is that in my recent work I have found that procedural logic far outweighs the amount of template text. The bigger reason is that most template languages don’t automatically protect you from script injection (XSS). There are many JavaScript template languages available now and I don’t have a need for any of them. But <angular/> is different — it is not a template language and it looks like it protects against script injection by default.

<angular/> is new but it looks very promising.

CSS Sprites vs. High Contrast Mode

Using positioned background images (a.k.a CSS Sprites) has a number of benefits:

  • Performance. One of the performance recommendations made by Yahoo’s performance team is to reduce HTTP requests. One way to do this is using the CSS Sprite technique — combining many images into a single background image.
  • Skin-ability. With the img tag the image URL is in HTML markup where it is hard to change. With CSS Sprites the image URL is in a CSS file making it more convenient to change. Sure you can change the contents of the original file but there are reasons for changing the URL. It is not as easy to move the images to a different server or group them in different image files when the URL is not in CSS.

However there are drawbacks. The biggest is accessibility, specifically high contrast mode. In high contrast mode all background images and colors are ignored — replaced with high contrast colors such as white on black or black on white. The other issue is that background images are not always printed.

The prevailing accessibility advice is to not use background images for functional images. The underlying problem is that there is no way in HTML/CSS to identify a background image as being functional. They are all considered decorative. It is also true that not all img tag images are functional but again there is no way to distinguish them for the purpose of high contrast mode. So high contrast mode makes a reasonable assumption that background images are decoration to be removed and img images are functional and must be shown. From here on I’ll call functional images icons. They either convey important information or are interactive (like a toolbar button for example).

I have seen recommendations that functional background images should be replaced with text when high contrast mode is detected. This does not seem right to me at all. A desktop app does not change its toolbar button icons to text in high contrast mode. The assumption is that icons are already designed to have reasonable contrast.

It also just feels right to me that the icon URLs should come from CSS.

Since I care about performance and accessibility I’m not happy with this conflict. I want a solution that puts the icon URL in CSS, works in high contrast mode and allows me to combine icons into a single image file. Here is what I came up with.
Continue reading CSS Sprites vs. High Contrast Mode

IE8 has strange rules for showing focus

While working on some accessible UI widgets I found strange behavior in IE8 related to when it does or doesn’t show the focus outline around an element that has focus.

When you use the focus method to set focus to a focusable element the expectation is that, in the absence of css rules to the contrary, the browser will show a focus outline around the element using its default style. This is what happens in Firefox, Chrome, and Safari. (Opera has its own set of focus problems which I didn’t investigate fully.)

There are a number of elements that are naturally focusable, such as inputs and anchors. In addition any element with a tabindex attribute equal to -1 is focusable. If the tabindex is non-negative then it is focusable and in the tab order. My test case uses list elements with tabindex of -1 or 0. I suspect that the same is true of other elements using tabindex.

Older versions of IE (I tested IE6 and I believe IE7 works the same in this regard) had some strange additional requirements beyond just calling focus. The outline would not be shown if the focus method was used to give focus to an element with a tabindex of -1. So before setting focus you would have to set the tabIndex to zero (or a positive number).

elem.tabIndex = 0;
elem.focus();

Also if you click on an element with tabindex of -1 no outline is shown unless the className is updated (even if the value doesn’t change). Perhaps other style changes would work as well.

IE8 adds some new bizarre behavior. First if css is used to set an explicit outline style on the elements (either directly or inherited) then the old IE quirks mentioned above go away as well as the new IE8 problem described below. Setting your own outline style for focus isn’t a good idea unless you are giving your whole web app a focus style makeover. I think it is best to stick with the default styles since it is what the user is probably used to. There is also the problem of old browser like IE6 not supporting the outline style.

If the className of the element receiving focus is updated in the onfocus handler while the tabIndex is -1 then no focus outline is shown. Assuming that the tabIndex is -1 and addClass is a function that adds a class to the className, one would expect the following two lines of code could be executed in either order.

addClass(elem, "myfocus");
elem.tabIndex = 0;

In IE8 setting the tabIndex to zero must be done first or the outline is not shown. This is exactly what tripped me up. If I just happened to write the statements in the opposite order I would never have discovered these oddities.

Note that in all cases focus is actually set correctly it is just the visual indication of focus that is wrong – but thats the important part of focus.

I created a test page that allows playing around with these rules. It is also an example of how focus can be managed with a roving tabindex. If you play with this page in different browsers you will notice that in Firefox (versions 2 – 3.5) , Chrome, and Safari 4 (I tested on Windows) it doesn’t matter what options are checked, the behavior is correct. In IE you can play with the options to see which combinations work.

I wonder if this is intentional in IE8 or is something that will get fixed in a point release.