Thoughts on APEX Progressive Web Apps

There has been a good deal of buzz around using APEX for Progressive Web Apps (PWA) lately thanks to the excellent work and presentations by Vincent Morneau. You can and should read his Turning APEX into a PWA document. Here I want to share my thoughts on building PWAs with APEX; the possibilities, limits and challenges. This post is less researched and more quickly written than most of mine but not really shorter. It is not based on any actual implementation or hands on investigation. As a member of the APEX development team I must emphasize that this contains my own opinions. Any forward looking statements about APEX are purely speculative and not part of any official statement of direction.

Vincent has pointed out that Interactive Grid and Interactive Report do not work offline and cannot be made to do so because they internally make ajax calls to the server. This is true but it is just the tip of the iceberg. The following are some of my bigger concerns.

APEX is multi-tenant and this works against the web’s same-origin policy. Consider apex.oracle.com, APEX sees it as 38,000 some odd workspaces and 100,000 plus web apps but the web sees one web app. APEX and the Oracle Database keeps all the workspaces and their schemas isolated. But on the browser there is no built-in separation. The browser considers all the apps from all the workspaces on an APEX instance to be one, which means it assumes they all trust each other. So browser features like named windows, Local/SessionStorage, and IndexedDB will let any app access the data of any of the others. This has serious security and privacy consequences. APEX has APIs such as apex.navigation.popup and apex.storage.getScopedSessionStorage to keep apps from stepping on each other but it is advisory only; it cannot be enforced. (Note you don’t have to use these browser features.) Because an offline PWA uses IndexedDB something will be needed to keep apps separate. Either some web-tier magic is needed so the apps are in their own origin or you have to deploy one app per APEX Instance. Perhaps there is something that ORDS can do to help. I don’t know. I should point out that this issue is not unique to APEX but would apply to any multi-tenant web platform.

PWAs require specific files to be at the root of the web URL. APEX doesn’t provide direct access or control over the web tier. It may be that creating a PWA will always require configuring the web server. Again ORDS may be able to help.

Vincent describes the “app shell” as the set of resources that need to be cached. This is where the APEX architecture works against the PWA in a fundamental way. APEX page resources mix data and static/structural presentation. Server side conditions and other logic affect what parts of a page are rendered. This means that the same page resource can be, and generally is, different each time it is requested. APEX pages are not a shell they are the whole egg (or turtle?). Think about a simple address book app. The address edit detail page as rendered by the APEX engine would contain input fields (page items) for things like name, address and phone number but these fields would also have values based on the current entry being edited. The field values are not part of the shell. In addition the page may be used for both edit and create in which case there is likely some server side conditions to choose between rendering a create button or an update button. If this page resource gets cached you have also cached data for a particular entry. When it is time to use this page offline you need to write code to replace the values with the actual entry data to be edited. Or you have to cached a copy of the page for each entry. If you go this route it is not really a shell that is being cached.

In contrast the architecture of modern JavaScript frameworks is quite different. Data and presentation are separate. The page resource is essentially static. The client makes a separate REST ajax request for the data. The data is stored and manipulated in a client side data model that is bound to the UI. This is a high level over simplification as there are many different frameworks, each with their own peculiarities. The main point is that separating the data from the presentation in this way makes it easier to implement a PWA. The page resource is static and suitable for caching. The client already has the logic for updating the UI so that it shows the current data. All that needs to be done to support offline is switch to get the data from indexedDB rather than an ajax request.

APEX has made some architectural changes in this area. In release 5.1 Interactive Grid introduced a client side data model. This separates the data from the presentation. The data is fetched from the server as JSON, stored and manipulated in the model, and presented and edited in the IG UI. The binding between the model and various views such as grid, single row, icon, detail and chart is handled automatically. (Note: the fact that the JSON can be part of the page resource is an optimization that can be turned off.) The APEX JET Charts also have a data model as this is a necessary part of using JET visualizations. I hope that this architecture is used more going forward.

Although currently IG cannot be used in a PWA mainly because of the ajax requests associated with saved report settings, architecturally it is in a better position than most other parts of APEX to support offline PWAs in the future. This is because of the separation of data and presentation. What needs to be done is to switch between the APEX server and indexedDB according to online status for saved report settings and to synchronize the settings. The local report settings would also be used when fetching offline data from indexedDB into the IG model layer. These are changes that APEX would have to implement. However the modular components that make up the Interactive Grid region such as the model and grid widget could potentially be useful in a PWA today. I say potentially because I have not tried it yet. The question is does the model API have the necessary methods for fetching and saving the data in indexedDB. I am very open to improving the APEX model layer in this area.

This sounds hopeful for the future of Interactive Grid but what about other parts of APEX? The model is designed to handle forms (a single record) and trees as well as reports. These are not currently exposed in any APEX region but you may be able to do it yourself. Keep in mind that for reports the model and tableModelView widget can do just about anything that Classic Reports can do and in a way that keeps the data and presentation separate. The IG Cookbook has an example on page IG Cards that demonstrates a card style report. This could also be done with a model and tableModelView cutting out the IG region to avoid its current PWA unfriendly behavior.

What is missing in APEX for general data and presentation separation is client side templates and data binding. Since 5.1 APEX has client side templates (see apex.util.applyTemplate) but they are as limited as APEX server side templates. One nice property of APEX templates is that they are logic-less and this is a property that I think should be maintained. Some people have used Knockout or other client side template and/or binding libraries. The problem with these in an APEX context as I see it is they require creating a model layer in JavaScript and because they are back-end agnostic require writing code to populate and store that model. This is not something that I think APEX developers should be required to do. The APEX model when integrated as part of an APEX plug-in is a declarative client side model that doesn’t require custom client or sever code. If APEX had richer template syntax that worked on the client and declarative data binding to the APEX model that would be a huge advantage for PWAs and would reduce the amount of dynamic actions needed for dynamic client side presentation updates.

Currently saving the APEX model only works with the Interactive Grid. I hope this changes in the future. Either by making it possible/easy to create your own DML process for your model based region plug-ins or not hard coding the Automatic Row Processing (DML) process to the Interactive Grid region. See the IG Cookbook Tabbed Record Editing page for an example that uses the model and simple two way data binding.

Hopefully I have made the advantage of separating data and presentation, having a client side data model and doing client side rendering, clear in the context of a PWA. Next I want to look at different kinds of data in terms of state. I like to divide the state data into two categories; conversational state and application or persisted state. Application state is what gets persisted in a database for reasonably long periods of time. For example, in an address book app the application state is all the addresses. It can also include users and their preferences. Conversational state is data that is needed while the user is interacting with the app. It is everything from current focus element, selection state, scroll offsets, current pagination offset, current active tab, collapsible region or tree control expansion state, to user entered data that needs to be remembered from one page to the next. An important feature of APEX that has been around since the beginning is session state. APEX keeps item values in session state so they are not lost due to the stateless nature of HTTP. Session state values can also be passed from one page to the next in the APEX URL. The distinction between application and conversational state is not always clear. Is a shopping cart conversational or persistent app state? It depends and can be argued both ways. The point here isn’t to nail down the definition exactly but to point out that JavaScript applications, especially single page apps (SPA), keep just about all their conversational state on the client. This means that when it is time for them to go offline they only have to worry about application state not conversational state. Because APEX apps keep most conversational state on the server extra work needs to be done to support offline use. I’m not sure if there is a general solution here. I think the best thing to do is to limit the use of APEX session state in a PWA app.

An issue related to session state is session state protection. APEX protects various values rendered in pages or in URLs from changing by using a checksum. For example consider a simple product purchasing UI. The user is shown the product, price, quantity, and total. The quantity is a number field or select list and the user can enter or choose a different quantity (all the other values are display only). There is client side code such as a dynamic action to recalculate the total when the quantity changes. When the page is submitted the server recalculates the total from the quantity and price given to it. Without session state protection the user could modify the price (by using the JavaScript console for example) and end up paying less than they should. The session state protection checksums let the APEX engine trust that data from the client has not been modified. The alternative would be to not use the price value that comes from the client (in fact don’t even submit it) but instead read it from the database again. Being able to trust the protected data that comes from the client improves performance by cutting down on extra reads from the database.

If protected values are stored locally such as in indexedDB while the app is offline even if the checksum is also stored it may not be possible to successfully save the data back to the server once the app is online again. It depends on if a new session is established when going back online. Buried in this whole concern over session state is the question just how will sessions be maintained and reestablished by APEX PWAs?

I gave a very simple example but checksums are used in many places in APEX including on URLs, URLs that open dialog pages, and interactive grid model data. How session state protection behaves or could behave when there is the possibility of going offline requires much more thought and investigation. Simply turning it off every where in your app just to support offline should not be done without carefully considering the security implications.

A JavaScript web app written in some other framework probably doesn’t have this session state protection checksum problem. Remember these types of apps get and save their data using REST resources. Hopefully these REST resources are implemented to not trust any data that comes from the client. So in this example the REST resource would only receive the quantity from the client. The price would be fetched from the database again to calculate the total. (In reality the price may be cached in some web-tier object store.) The point is that this is another example of how other architectures may have an easier time supporting offline compared to APEX.

My final though is synchronization is hard. Simply saving and replaying ajax requests is not data synchronization. If you add a contact to your address book while offline and then delete it while still offline, it makes no sense, once back online to send a request to add and then another to delete the contact. The same is true if contacts are edited multiple times. The database just needs to be synchronized to the final state. And the synchronization goes both directions updating the server with changes made while offline and also updating the client with the latest data from the server. I also feel that in an APEX app you shouldn’t need to define separate web services to support offline use. There is no reason why ajax calls can’t be made to APEX. Ideally this would leverage the normal APEX ajax code paths. If using the APEX model this includes validation. A big part of the app design for offline is going to be deciding what data to make available offline. It is probably unwise to replicate the entire database into indexedDB. Perhaps just the user’s data, or just the data from the last few days or weeks, or just data related to a specific project etc. The user may choose what to make available offline or perhaps whatever they have been looking at most recently is transparently written through to indexedDB. There are so many possibilities. When you do synchronize there can be conflicts. APEX already has built-in support for detecting conflicts using either row values or a version column. However the longer the time between fetching the data and saving it means the more chance for others to make conflicting edits. It will be even more important to provide ways for the user to correct the conflicts and save without having to completely redo all their changes.

Many of my concerns have been about offline use but as Vincent points out offline is only one piece of what it means to be a PWA. The main aspects of a PWA are:

• Installable: The app can be installed and resources cached for better performance. This is progressive meaning that the web app still works without being installed or on browsers that don’t support PWAs. Being installed makes the web app feel more lake a native app with features such as a home screen icon, full screen etc.
• Responsive: The app works well in different form factors from phone to desktop. Universal Theme is already good at this. There are specific region types like the Reflow Report designed to be responsive and ideally we will continue to improve responsive capabilities throughout.
• Offline: The app can still work when there is no network connection. How much of the app can be used offline is going to depend based on app purpose, user needs, and effort to make it work. There is always going to be some effort involved in making this work. It is not clear to what extent APEX can make this easy/declarative and for which use cases.
• Notification: Web push and notifications API allow the user to receive notifications from the app even when it isn’t running. APEX has good support for sending emails but who uses email these days? Well I still do but some users may prefer this.

When you are telling the APEX team you want PWA support be specific. Let us know what aspects of PWA technology you need the most.

So will this happen; will APEX support some or all aspects of PWA technology? If so which and when? Should we even bother? I hear some people say this isn’t even something that APEX needs to support; let other frameworks worry about that. Often it is a specific statement about offline and how the world is getting ever more connected. This may be true but there are exceptions. The web site Mountain Project is a good example of where offline is useful. It is a guide to outdoor rock climbing and many rock climbing areas do not have Internet access. It is not a PWA but instead has an app for Android and iOS. It could have gone the PWA route I suppose. Before you head out to the crag you download the area of interest and then the app works while offline giving you all the climbing route info.

I have no idea to what extent if any APEX will embrace PWAs but I do have an opinion. I think that it should if for no other reason than I believe that APEX should be able to do anything that other web platforms/frameworks can do. It could be that in the future being a PWA is simply what is expected when you claim to have good mobile support.

Let us know your Progressive Web App use cases and needs and how you would like to see APEX support them.