APEX Template Directives

In APEX 20.2 you will see in the help text for a few properties “Additional Information… Supports Template Directives”. Here I will explain all about this new functionality, the motivation, where to find it, how it can be used, where it may be going, and hopefully clear up and establish some terminology.

Background

APEX has always had (at least as far back as my familiarity with APEX goes) something called templates. You can find all the templates under shared components. There are a number of kinds: Page, Button, Region, and Report to name a few and also Email Templates. At the same time, the term templates and template processor has a broad but well understood definition in the area of computer science. There are many examples of template systems across various platforms, such as Mustache, Handlebars, JavaServer Pages and one of my favorites StringTemplate. If you have experience with one or more template systems you may have found APEX templates lacking functionality or maybe just something a little different.

Here is a simple and broad definition: A template processor is a system that takes as input a set of data and a set of template text and combines them to produce output text(s). This fits APEX but lets look a little deeper.

A template in APEX is a related set of metadata attributes most of which are HTML text snippets that support substitutions. A substitution is a token or placeholder syntax such as #ROWNUM# that is replaced with a data value at runtime such as 257. Depending on the template type there may be other metadata attributes such as row conditions, related JavaScript or CSS, and “template options” which just provide a declarative way to mix in specific CSS classes to the HTML snippets via substitutions. Here I’ll use the term APEX Template to refer to these sets of metadata to avoid confusion with the more general term template.

The current syntax for placeholders hasn’t changed since the beginning but data substitution syntax has evolved slowly over time. Very early syntax was: &ITEM. Then a closing period was added to make parsing more robust &ITEM.. Later quote syntax was added to support international characters &"ITEM".. To support context specific control over escaping the format specifier was added, for example &ITEM!HTML. and most recently text message substitution was added &APP_TEXT$MESSAGE_KEY..

Although the placeholder substitution syntax is common and there is a common internal function for handling some of the substitutions such as application items or built-in substitution strings, there is no general template processor or engine in APEX. Each of the APEX Template types is processed by code specific to the task at hand and it is in full control over the way in which the individual HTML text snippets are assembled and the substitutions that are supported.

In a typical template processor system each of the metadata attributes that I called a HTML text snippet would be called a template. In fact, throughout APEX, every declarative setting/property (metadata attribute) that supports symbol substitution is in fact a template (although we don’t call them that). This is because the simplest type of template processor just supports placeholder substitution. The new JavaScript ES6 Template Literals are an example of such a simple template.

But for many developers templates don’t really deserve the name until they support more advanced features. (They may prefer the term string interpolation for these simple substitutions.) Most template processors support the following features:

• Substituting a placeholder for a data value.
• Conditionals to output one part of the template or another depending on data values.
• Looping over some kind of data sequence, list, array, or collection.
• Inclusion of one template inside another. This is for modularity and reuse and sometimes includes passing arguments like a function call.

They vary greatly in the details and the syntax used to express the above functionality. One important detail is how much embedded logic the templates support. They range from no logic at all to allowing the full computational power of the host programming language or may include their own expression language. The term logic-less templates is used to describe template processors with little or no logic. Volumes have been written on this hotly debated topic. I won’t go into details on this but the general motivation for logic-less templates is the idea that business logic should be kept separate from presentation details like HTML markup. My personal preference is for logic-less templates.

When I first joined the APEX team and learned APEX I was a little disappointed with the limited functionality of APEX Templates but I was very pleased that they allowed absolutely no logic. In my mind this meant they could potentially be enhanced and didn’t need to be “fixed”.

It may seem like there is a contradiction here. How can templates support conditions, looping and function call like inclusion and still be logic-less. These sound like things a general purpose programming language have. What logic-less means is that the templates have no expressions or comparison expressions and can’t change the data in any way or create new data. Templates can’t do any math or arbitrary comparisons. Templates have no side effects. So even the simplest expressions like A + B > C are not allowed.

Clearly producing a whole web page or even a list requires conditions and looping. These things are not explicit in the APEX Templates, which only support substitutions. APEX handles this by putting the logic in the engine. Consider a list template. First the template is mainly concerned with the items. APEX handles the list data and implicitly loops over it processing the appropriate HTML snippet for each item. This means when the HTML snippet uses #LINK# it is referencing the link URL for the current item in the loop. There are a number of fixed cases for list items: are they the current item, are they at the top or sub level, do they have sub items, is it the first item in the list. APEX handles this by having different HTML snippets for each case. The APEX engine decides which snippet to process for each item.

The benefit of APEX Templates is that it keeps the template syntax very simple. This means simple to learn, read, write and reason about. It also means simple to parse and process and this translates into having good performance as well.

There are a few down sides. One is duplication. Taking the APEX list template as an example again you can see that there are often only slight differences between the different templates such as current and non-current. This results in much duplication of markup and could be solved with simple conditions. There is also duplication between APEX Templates for example there is plenty of consistent markup between Page Templates that could be shared if there were a way to include or call template fragments.

Another negative is the fixed set of conditions and associated HTML Fragments sometimes require tricks to achieve the desired result or impose limitations that we live with. Just a few examples:

• The Universal Theme Navigation Bar list template doesn’t allow sub menus because list templates have only two levels, list and sub list.
• Many APEX Templates generate excess markup. This results in pages that are a little larger than they need to be. In some cases CSS is used to hide what is not needed. APEX List Templates for navigation tend to have many attributes with empty values. For example data-icon="#ICON_CSS_CLASSES#". If there is no icon this results in an empty attribute when it would have been better to omit the attribute altogether.
• APEX Report Templates support up to 4 different row templates with their associated condition and expression. Four is generally enough but what if you needed more? Also the expressions tend to tie the Report Template closely to the report; something that would be good to avoid.
• Some of the APEX List Template HTML snippets such as Template Definitions for First Entry are remnants of a Web long gone.

Solving any of these specific problems has not been and still isn’t much of a priority. In fact there has been a steady move away from APEX Templates. This doesn’t mean that they are not used or not important or going away. It is a matter of who has to edit them and how often. It used to be that APEX had many themes each with their own set of APEX Templates. It was more common for people to modify theme templates. Now with Universal Theme, APEX provides a single comprehensive responsive theme. Yes it changes over time but the changes offer continuous improvement and enhancement rather than something completely new that apps must adapt to. In addition, recent new region types have not introduced new template types. Instead they handle their own rendering. The general idea is that rich presentation and behavior is difficult to get right. Getting it right means it is usable, secure, and accessible. Allowing customers to customize the markup via templates is hard to support and makes it too easy for them to make mistakes.

It is now very rare to have to create or edit an APEX Template. Most low-coders will never need to touch them. Here are some cases where you might:

• You are creating your own custom theme. This is a huge undertaking and is almost never done because The APEX Universal Theme already does a very good job.
• Creating email templates is required since there are no predefined email templates.
• List and Report templates sometimes need customizing. A best practice is to start with a copy of a Universal Theme template that is close to what you want and then make changes. Before doing this consider if there is a template option or other custom CSS rules that can achieve what you want.
• Occasionally a custom region or page template is needed and again it is best to base it off of a Universal Theme supplied template. I have done this to create a jQuery UI Tabs region and a Page and Region combination for the undocumented APEX splitter widget (see the IG Cookbook).

Another reason it is less likely to need to change templates is that you can do so much more with CSS now. Remember the old days of table layout, images for round borders, inline styles in markup etc.

Now the most common places where even a low-coder may need a little bit of HTML markup is in individual properties that support substitutions. A common example are the HTML Expression properties available for various report columns including Interactive Grid, Interactive Reports, Classic Reports and Cards.

Motivation

Even thought it can be argued that APEX Templates are good enough and don’t need enhancing there have been requests over the years, both internal and external, for new template facilities and/or enhancements to APEX Templates.

Developers familiar with what other frameworks do have requested that APEX include built-in support for <insert name of favorite template library here>. It is often related to client side rendering where the server sends just the data to the client as JSON and the client renders the markup. You can see an example of this type of request and instructions on how to use Mustache here. This client side rendering of JSON is a direction APEX has already started to move in with Interactive Grids and now Cards.

We hear a general complaint about how its cumbersome to use APEX Report templates because the SQL query column names need to match the Report Template placeholder names. This results in tight coupling between the Classic Report Region and the Template. This issue was getting a lot of attention as we were looking at making it easier to have a Faceted Search + Cards UI.

A specific enhancement request we hear often is that APEX Email Templates could really use “if” conditions. Sometimes everyone gets the same email and sometimes you want to tweak the message just a bit.

Another source of frustration is HTML Expression properties and handling NULL values. Many regions have a “Show Null Values as” property but this doesn’t apply when the column uses an HTML Expression. The HTML Expression is still used but it would be nice if it could give a NULL value special treatment.

There are various specific ways to satisfy each of the above requests/complaints but a general enhancement to templates (here templates mean properties that support symbol substitution) provides a road map to potentially satisfying them all and more.

The main motivation for template enhancement was the new Cards region. We choose to create a new Cards region because it could be done faster and easier and also provide an easier to use and more powerful solution for our customers than trying to enhance the existing Classic Report using APEX Report Template solution. The Cards region could have been done without any template enhancements but having them made some of the internal implementation details easier for us and also exposed very useful functionality for our customers, which I will soon show.

Our starting assumptions for any template enhancements were:

1. Templates are processed on client and server using the same syntax and feature set. This has been true since APEX 5.1. Since then the client has kept pace with server changes. When the APP_TEXT$ syntax was added on the server to reference text messages the client added support for the same syntax. This doesn’t mean the client and server have to be in sync in every release but, for example, the server shouldn’t invent a new way to do the same thing that was added to the client in a previous release.
2. Extend what we have. There is no green field opportunity here. The current placeholder and symbol substitution has to keep working. (There is one minor change in behavior noted below.)
3. Keep it simple. This means easy to implement, fast to process and also easy for customers to use and understand.
4. Continue to be logic-less. There are two good reasons for this. As already mentioned there is value in keeping business logic out of markup. Adding expression capability would go against the desire to keep it simple.

Taken as a whole these ruled out using any third party library. Any enhancements would need to be APEX specific.

Template Directives

Finally, its time to talk about the new 20.2 feature, template directives. Template directives are special tokens in a template that control how the text around it is processed. Here template means the value of any declarative property that supports substitutions and template directives. To avoid confusion with APEX Templates we don’t call them templates in the help or documentation. They are just properties (also known as settings or attributes) that support substitutions and directives.

The general syntax for a template directive is:

{directive-name arguments/}

The arguments are optional and depend on the specific directive. Directive names are case insensitive. The directives are: if, elseif, else, endif, case, when, otherwise, endcase, loop, endloop. There are also comments and, for rare cases, a way to escape a lone open curly bracket. I won’t go into all the details here. You can read all about the directives in the JavaScript documentation for apex.util.applyTemplate and also an overview in section Using Template Directives in App Builder User’s Guide.

Currently template directives are only supported for substitutions done on the client by the applyTemplate function. This means they are limited to Interactive Grid and the new Cards report regions. Both of these regions do the HTML rendering on the client. Basically any property that gets processed by applyTemplate supports directives. We added the “Supports Template Directives” note to the help for a number of Card and Interactive Grid properties but didn’t get them all.

Here is an example of how this new feature can be used. The following shows a few cards from a new Cards region on the DEMO_PRODUCT_INFO table from the Sample Database Application.

The interesting part is in the body of the card. Notice that there is variation in each card. Some show just the description, some show a warning with an icon if the product is out of stock and some a nice list of tags associated with the product. This is easy, right, we APEX developers know how to get the job done with SQL. It goes something like this

In the SQL Query put conditional HTML markp:

select
    ...
    PRODUCT_DESCRIPTION,
    case when product_avail != 'Y' then
        '<span class="fa fa-sm fa-exclamation-triangle-o" aria-hidden="true"></span>Out of Stock!'
    else null end STOCK_WARN,
    '<ul class="a-tags">' || case when tags is not null then
        '<li class="a-tag">' || 
        apex_string.join(apex_string.split(tags, ','), '</li><li>') || 
        '</li>'
    else null end || '</ul>' TAGS_LIST,
    ...
  from DEMO_PRODUCT_INFO

Then in the Card Body HTML Expression use these simple substitutions.

&PRODUCT_DESCRIPTION.
&STOCK_WARN!RAW.
&TAGS_LIST!RAW.

Easy, and before 20.2 necessary, but there are at least three problems with this. First, there is presentation in the business logic. The SQL would be much easier to read and maintain without all those HTML elements. The person writing the SQL needs to know HTML (be aware of front-end best practices, usability, accessibility etc.) and coordinate the classes with the UI designer that will add the CSS rules.

Second, and I’m no expert on this, but I believe putting markup in your SQL can be bad for performance.

The last problem has to do with using !RAW. This is needed because the columns contain markup and without it the markup would be escaped as part of substitution. But do you see my error? I forgot to escape the tags column so now there is a likely an XSS vulnerability. It is best to let APEX do its default escaping to help keep you safe.

Now lets see how template directives let you create custom cards while keeping your SQL clean. Here is the SQL.

select
    ...
    PRODUCT_DESCRIPTION,
    PRODUCT_AVAIL,
    TAGS,
    ...
  from DEMO_PRODUCT_INFO

It doesn’t get any simpler than that. And the Body HTML Expression is:

{! &PRODUCT_AVAIL. &TAGS. /}
&PRODUCT_DESCRIPTION.
{if !PRODUCT_AVAIL/}
<span class="fa fa-sm fa-exclamation-triangle-o" aria-hidden="true"></span>Out of Stock!
{endif/}
<ul class="a-tags">
{loop "," TAGS/}
 	<li class="a-tag">&APEX$ITEM.</li>
{endloop/}</ul>

The first line is a comment and I’ll come back to why its there in a moment.

The if directive says that if the product is not available output the icon markup and out of stock message. The PRODUCT_AVAIL column contains either ‘Y’ or ‘N’. Any value that is empty or null or one of the characters ‘NnFf0’ is considered false by the if directive and anything else is true. The exclamation (!) in front of PRODUCT_AVAIL means not, so when PRODUCT_AVAIL is ‘N’ then the following text up to the matching endif directive is output.

The loop directive takes a value, in this case the TAGS column which contains an optional comma delimited string of tag values, and splits it up into a list and then evaluates the text between the loop and matching endloop directives once for each item. During each iteration the special symbol APEX$ITEM takes on the value of the current list item value. In this way each tag is wrapped in a LI element. As you can see there is no need for the !RAW format and APEX takes care of escaping each tag value.

Hopefully this example gives you a good idea of how directives works. You can read the documentation for details on these directives and the case directive that I didn’t show here. Even though this is documented in the JavaScript API documentation there is no requirement to know or use JavaScript in order to use these directives in the declarative attributes where they are supported. The semantics should be familiar to programmers experienced with procedural languages but also easy enough for non-programmers to learn. The directives can be nested as you would expect.

If you make a mistake in how you use the directives there will be an error in the JavaScript console and the developer toolbar will show the red error triangle icon. For example if I forget the closing / in the endloop directive ( {endloop} ) I will get this error in the console:

applyTemplate missing 'endif', 'endcase', or 'endloop'

Getting back to the first line of the HTML Expression. It is a comment. Its handy to be able to document your templates. Comments begin with {! and end, like all directives do, with /} and must be all on one line. (You could think of ‘!’ as being the directive name.) This comment is clearly not for humans. It is an unfortunate but necessary trick that will hopefully not be needed in the future. The data for client template processing mostly comes from items and APEX models. In this case it is the model data in the Cards region. (The Cards region has similar architecture as IG including the model.) But the Cards region doesn’t have column configuration and not all columns in the SQL result set necessarily make it into the client model. On the server side the Cards region detects which columns are used in HTML Expressions and makes sure those are included. When your template only references a column in a directive (meaning without the &COL. syntax) the Cards region doesn’t recognize that as a use. Notice how PRODUCT_AVAIL and TAGS are never substituted, they are only used in directives. But the detection code also doesn’t know about directives including comment directives so it finds the columns in the comment and that is how that data gets added to the client model without having to be rendered. This trick is not needed in Interactive Grid templates because all columns are included in the model. This trick can also be used if you have other reasons to want data in the model that is not used in the cards.

Undocumented stuff

Earlier I mentioned inclusion, or the ability for one template to call another, as one of the key features of most template processors. The new template directive documentation doesn’t mention anything like this but if you dig into how Cards work internally you will see that there are additional directives. Type this into the console:

 apex.region("productCards").call("option").recordTemplate

when you have a Cards region with Static ID “productCards” and you will see

{with/}
…
{apply APEX_CARDS_CARD_nnnnnn/}"

The argument to apply is the name of a template and all the stuff in between the with and apply directives are arguments to pass to the template. You will recognize parts of the argument values as properties you configured in your Cards region.

The reason these directives are not documented is that currently there isn’t much you can do with them without the ability to define a named template and there may still be changes made to the syntax or semantics. If you want to see the content of the template that is applied just pass the name of the template to the undocumented apex.util.getTemplate function. For example:

apex.util.getTemplate("APEX_CARDS_CARD_803086008591378417")

This is enough for a curious person to figure out and actually do something with these directives but you have been warned that things may change.

More applyTemplate goodies

In the past APEX has used suffix naming conventions to access data related to an item or column. The main example of this is the “_LABEL” suffix added to columns to access the column heading label. This kind of naming convention is not a good idea because it can be confused with actual columns or items; it pollutes the “namespace”. Also new in 20.2 is the substitution property reference syntax. A property reference starts with percent symbol (%) and follows the item/column name. See the documentation for the supported properties. Note that column properties only apply if the columns have configuration such as in Interactive Grid (but not in Cards).

So now in Interactive Grid you can use &ENAME%label. rather than &ENAME_LABEL.. The latter still works but the new syntax is clearer so you should switch to it were available. Interactive Report does substitution on the server so there you must still use the “_LABEL” suffix.

Back in APEX 5.1 when client side substitution was first implemented there was the question of what to do with substitutions that have a display value. For example the JOB column defined by a list of values has an item with value ‘CLERK’ and display value ‘Clerk’ (in English). The server always uses the session state value; it knows nothing about the display value. So on the server &JOB. gives you ‘CLERK’. But on the client in the context of Interactive Grid it seemed generally more useful to use the display value. So &JOB. gave you ‘Clerk’. This deviation from how the server worked was not ideal but it was practical and the only other solution at the time would be to add yet another naming convention such as &JOB_DISPLAY. which I really didn’t want to do.

Now with directives it is just as likely that you want the value of an LOV column, for example when used in a condition directive. With the new property reference syntax we could get the server and client semantics back in sync. We decided that the change in behavior was worth it to make the client and server substitutions consistent so in 20.2 client symbol substitutions no longer uses the display value. This change in behavior was supposed to be in the 20.2 release notes but it didn’t make it and will hopefully be added when the release notes are refreshed.

What this means is that if you were using symbol substitution and expecting the display value you now need to add %display property reference. For example change

&JOB. to &JOB%display.

For APEX plug-in developers, when you use apex.util.applyTemplate, the developers using your plug-in can automatically take advantage of the new directives and property reference syntax. Directive processing is turned on by default. If you don’t want directives to be used you can turn it off by setting the directives option to false.

Notes on using applyTemplate

One problem that you will likely run into when using applyTemplate is that the server will substitute symbols on the server so there is nothing left for the client to do. (The server is greedy.) As a simple example lets say you want to use applyTemplate in a dynamic action Execute JavaScript Code action. You might try this:

$s("P1_NAME", "Alex");
console.log(apex.util.applyTemplate("Hi &P1_NAME."));

The item P1_NAME is a hidden item that has no initial value. You expect to see “Hi Alex” in the log but instead you see “Hi “. If you look at the page source you will see that the dynamic action code is:

$s("P1_NAME", "Alex");
console.log(apex.util.applyTemplate("Hi "));

The server performed the item substitution so there was nothing left for the client to do. You do not have any control over this substitution. There is no setting or template syntax to turn off server side substitution. One ugly workaround you can use goes something like this:

$s("P1_NAME", "Alex");
console.log(apex.util.applyTemplate("Hi ~P1_NAME.".replace(/~/g, "&")));

The server does not recognize ~P1_NAME. so passes it to the client as is. The call to replace changes it to &P1_NAME. just before it is passed to applyTemplate.

Hopefully in the future there will be better ways to handle this but in the mean time the best option is to create a plug-in. As a plug-in developer you have control over substitutions for your own custom attributes. Just set Substitute Attribute Values to Off and then you can choose what to substitute on the server and what to pass to the client. See APEX_PLUGIN_UTIL.REPLACE_SUBSTITUTIONS

Future possibilities

The last topic is about some of the ways templates may be expanded or improved in the future and how it could benefit APEX developers. I must stress that even though I work on the APEX team these are just my thoughts on this mater and not part of any official plan or road map.

Template processing with the placeholder and substitution syntax started on the server and then moved to the client. This is the first time new template functionality was added to the client first. I hope that someday the server will support the same directives. This would mean that just about anywhere symbol substitution is supported template directives would also be supported. This includes in the HTML snippet properties of APEX Templates. I don’t think APEX Templates need any other major changes. Just having conditions will allow us to improve the Universal Theme Templates so that they generate smaller, simpler HTML. The directives would be a welcome improvement to APEX Email Templates where people have asked for this kind of thing since they first came out. It would also allow HTML Expressions to handle NULL column values with ease. Perhaps most importantly it would enable developers to move markup out of SQL in many more cases.

Ideally there would be a public server side API that supports template directive processing. It could be an enhancement to the existing APEX_PLUGIN_UTIL.REPLACE_SUBSTITUTIONS or something different. It doesn’t need to be like applyTemplate, just support the same syntax to the extent that it makes sense.

In order to take advantage of the with and apply directives there needs to be a way to define named templates. It is not clear what form this could take or even if it is necessary. It is possible that built-in named templates could be supplied as part of a theme.

It has been suggested that beter UI for entering template directives could make them easier to use. This could be as simple as syntax coloring and code completion in the APEX code editors or some kind of visual template builder.

Data binding is a distinct feature that often goes along with template processing. I shared some thoughts about this before and hope to have more to say in the future. The idea is that if you have a template that depends on some data value and that value changes the template should be reapplied. Currently this kind of UI update has to be done with dynamic actions involving JavaScript code and it can get quite complicated. A future APEX data binding feature that leverages template directives could make many kinds of dynamic UI updates possible with zero JavaScript.

As more template processing is done on the client, it may become necessary to give more control over, or be smarter about how the client and server coordinate template processing.

I can’t wait to see the fancy cards UI that APEX developers create. Also we would love to hear your feedback on this new functionality.