On-Demand Javascript

Time your website with WebWait - from the creator of AjaxPatterns.org

From Ajax Patterns

(Difference between revisions)

Revision as of 11:53, 6 December 2006

Archived Version
There's an archived version of this pattern available, taken from the Ajax Pattern book draft, showing roughly how it appeared before the page became publicly editable.

Evidence: 2/3

Tags: Behaviour Bootstrap Dynamic Javascript LazyLoading OnDemand ScriptTag

In A Blink

Javascript being pulled from the server

Technical Story

Devi's concerned about initial user experience, because her new website requires lots of Javascript. She refactors the application, extracting out a few Javascript modules. The initial page load is now much faster, because only a bare-bones initialisation module is pulled down, with the other modules loaded only when they're needed.

Problem

How can you deploy lots of Javascript code?

Forces

Ajaxian applications make heavy use of Javascript. Richer browser behaviour means more script code must be downloaded.
Downloading Javascript has a performance impact. Interaction cannot fully begin until all initial Javascript has been loaded.
Bandwidth is also a concern with heavy Javascript content. Often, not all Javascript that's loaded is actually used, leading to a waste of bandwidth.

Solution

Download Javascript as and when required, instead of downloading it all on page load. This is a "lazy loading" approach applied to Javascript, and has several benefits:

Initial page load is faster.
Overall bandwidth usage is less, since only Javascript that's required is used.
Deployment is easier as the code takes care of pulling in Javascript, with the coder only ensuring enough is there to kick the process off.
You can produce snippets of Javascript on the fly - effectively sending back a kind of "behaviour message" advising the browser on its next action. This is a variant of the core pattern described at the end of this section.
You can bypass the standard "same-domain" policy that normally necessitates a Cross-Domain Proxy. Provided a server exposes a segment of valid Javascript, it can be extracted by the browser.

Conventionally, best practice has been to include Javascript unobtrusively - by including it in one or more script tags.

  <html>
    <head>
      <script type="text/javascript" src="search.js"></script>
      <script type="text/javascript" src="validation.js"></script>
      <script type="text/javascript" src="visuals"></script>
    </head>
    ...
  </html>

On-Demand Javascript builds on this approach to suggest just a minimal initialisation module in the initial HTML:

  <html>
    <head>
      <script type="text/javascript" src="init.js"></script>
    </head>
    ...
  </html>

The initialisation module declares whatever behaviour is required to start up the page and perhaps enough to cover typical usage. In addition, it must perform a bootstrapping function, pulling down new Javascript on demand.

So how do you load Javascript after the page has loaded? One way is with an XMLHttpRequest Call. The browser can easily accept a Javascript response and evaluate it. Any code not inside a function will be executed immediately. To add new functionality for later on, the Javascript can add directly to the current window or to an object that's already known to exist. For example, the following can be sent:

 self.changePassword = function(oldPassword, newpassword) {
   ...
 }

The callback function just needs to treat the response as plain-text and pass it to eval(). A warning: here again, asynchronicity rears its ugly head. You can't assume the new function will be available immediately. So don't do this:

 if (!self.changePassword) {
   requestPasswordModuleFromServer();
 }
 changePassword(old, new) // Won't work the first time because
                          // changePassword's not loaded yet.

You either need to make a synchronous call to the server, add a loop to keep checking for the new function, or explicitly make the call in the response handler.

A less obtrusive way to do the same thing involves DOM Manipulation. Simply insert a suitable script element into the DOM and the Javascript will be loaded. It can be a child of either the head or body and its src property must be pointing to a Javascript URL. There are several differences with XMLHttpRequest:

The Javascript will automatically be evaluated in much the same way as Javascript linked in the static HTML is evaluated when the <script> tag is first encountered. You can declare a bare function or variable without adding it to the window.
You can load Javascript from external domains.
The URL points to a specific Javascript resource. With XMLHttpRequest, there's more flexibility: you can, for example, send several Javascript snippets inside different XML nodes.
The DOM is affected in a different way. Even if the behaviour is transient, the script will be added to the DOM, whereas it would disappear after an XMLHttpRequest callback eval'd it.

(Watch for compatibility. See http://www.howtocreate.co.uk/loadingExternalData.html)

There's a variant of the pattern I'll call Javascript Response in reference to the HTML Response pattern. The standard pattern refers to the situation where you're downloading a new set of functions, classes, and variables. In reality, you're just adding new members to the DOM - some other code will then invoke it. In Javascript Response, you grab a snippet of Javascript and eval it - the script is not wrapped inside a function (although it may define some functions that it calls). Under this variant, then, the browser is asking the browser what to do next. Whereas in the standard pattern, the browser knows what to do, but not how to do it.

One important application is with JSON Messages. Being standard Javascript, they can be downloaded using On-Demand Javascript, which is a convenient way to transfer semantic content, i.e. a self-contained object. Because Javascript can be pulled from third-party sources, this provides a mechanism for transferring semantic content from an external server into the browser, and without a Cross-Domain Proxy. Some Web Services, designed for public browser-based usage, therefore output JSON Messages.

Decisions

How will you break modules down?

You'll need to decide how to carve up your Javascript. Standard principles of software development apply: a module should be well-focused, and inter-module dependencies avoided where possible. In addition, there are web-specific concerns:

Ideally, any given module is either not used at all, or used in entirity. What you don't want is a 5000-line module that's downloaded to retrieve a 3-line function. It's a waste of bandwidth that defeats the main purpose of On-Demand Javascript.
You need one or more modules present on startup. At least one is required to kick off further downloads as required.

Flickr's Cal Henderson offers some useful guidelines on this. TODO Expand

How will the script determine which Javascript to download?

The script-downloading mechanism itself is quite easy, but determining when you need to download requires some consideration. Imagine you have a Fat Client numerical modelling website, with hundreds of algorithms available to be downloaded whenever the user requires them. Based on user input, how do you decide if you need to download something, and if so, what? Maintaining a long switch statement is going to get pretty tired - you ideally want a solution that's as unobtrusive as possible. MapBuilder, discussed in the examples, for instance, integrates Javascript loading into the overall application configuration.

At what stage will the script download the Javascript?

It's easiest to download the Javascript just before it's required, but that's not always the best approach. There will be some delay in downloading the Javascript, which means the user will be waiting around if you grab it at the last possible moment. When the user's actions or the system state suggest some Javascript will soon be needed, consider downloading it immediately.

Predicting if Javascript is needed is an example of Predictive Fetch - grabbing something on the hunch that it might be needed. You need to make a trade-off about the likelihood it's required against the hassle caused if you hold off until later. For example, imagine you have some Javascript to validate a completed form. If you wait until the end, you'll definitely not be wasting bandwidth, but at the expense of the user's satisfaction. You could download it when there's one field to go, or two fields, or when the form's first loaded. Each option increases the chance of a wasted download, but increases the chance of a smoother validation procedure.

Real-World Examples

MapBuilder

MapBuilder is a framework for mapping websites. It uses On-Demand Javascript to reduce the amount of code being downloaded. The application is based on a Model-View-Controller paradigm. Model information is declared in an XML file, along with Javascript required to load corresponding widgets. When the page starts up, only the required Javascript is downloaded, instead of the entire code base. This is therefore a partial implementation of this pattern; it does ensure that only the minimal subset of Javascript is ever used, but it doesn't load pieces of the code in a lazy, on-demand, style.

Dojo Package System

Dojo is a comprehensive framework aiming to simplify Javascript development. As such, it provides a number of scripts, of which an individual project might only use a subset. To manage the scripts, there's a Java-like pacakage system, which lets you pull in new Javascript as required. You only need include a single Javascript file:

  dojo.hostenv.moduleLoaded("dojo.js.*");

You then pull in further packages on demand like this:

  dojo.hostenv.moduleLoaded("dojo.aDojoPackage.*");

will download all relevant Javascript modules under dojo.aDojoPackage. The precise set of modules is defined by the dojo.aDojoPackage author, and can be made dependent on the calling environment (whether browser or command-line).

JSAN Import System

The JavaScript Archive Network (JSAN) is an online repository of scripts. As well, it includes a library for importing Javascript modules, also a convention similar to Java. The following call:

 JSAN.use('Module.To.Include');

is mapped to Module/To/Include.js. JSAN.includePath defines all the possible top-level directories where this path can reside. If the includePath is ["/", "/js"], JSAN will look for /Module/To/Include and /js/Module/To/Include.

twoBirds full AJAX

This AJAX system is providing on-demand web-object functionality. In addition to load the .js file on demand, it also loads .html.tpl and .css files dynamically into the browser cache structure. A given twoBirds web-object has an init() function, that checks whether all files needed are present in the cache, if not waits for them to be loaded, and then calculates the html code of the element to be included in the DOM model. The following call:

tbElement.show( 'target div id', '<module>', '<element>' );

will load and display an element on the webpage in the given DIV element. twoBirds includes a modular site structure based on module directories.

Since a twoBirds web-object may include other twoBirds web objects, the site overall complexity level is unlimited, still allowing quality management of the code.

twoBirds is not yet out in the open, see [1] for an early prototype. Contact the programmer if you want to get a copy of the current stage of development.

Refactoring Illustration

Introducing On-Demand Javascript to the Wiki Demo

In the Basic wiki Demo, all Javascript is downloaded at once. But many times, users will only read from the wiki - why download the code to write to it? So this demo refactors to [On-Demand Javascript], in three stages:

1. The uploadMessage function is extracted to a second Javascript file, upload.js. 2. A further refactoring pulls in the upload.js file only if and when an upload occurs. This version uses the DOM-morphing approach. 3. In yet another refactoring, the DOM-morphing approach is substituted with the XMLHttpRequest approach.

Separate Javascript: Extracting `upload.js`

In the first refactoring, the upload function is simply moved to a separate Javascript file. The initial HTML includes the new file.

  <script type="text/javascript" src="wiki.js"></script>
  *** <script type="text/javascript" src="upload.js"></script> ***

The new upload.js now contains the uploadMessage function. Reflecting the separation, a couple of parameters were introduced to decouple the function from the main wiki script.

 function uploadMessages(pendingMessages, messageResponseHandler) {
   ...
 }

The calling code is almost the same as before:

 uploadMessages(pendingMessages, onMessagesLoaded);

We've thus far gained a bit of modularity, but no boost to performance, since both files must be downloaded on startup.

DOM-Based On-Demand Javascript

This refactoring and the next both achieve On-Demand Javascript, in different ways. Neither approach is intrinsically superior - each approach has its own characteristics, as discussed in the Solution above.

With On-Demand Javascript, the upload.js is no longer required on startup, so it's reference no longer appears in the initial HTML, leaving just the main module, wiki.js.

 <script type="text/javascript" src="wiki.js"></script>

To this file, a new function has been added to download the script. To avoid downloading the script multiple times, a check is made against uploadMessages, the function that's downloaded on demand. It adds a script element to the document's head, initialised with the upload.js URL and a standard Javascript type attribute. This new DOM element is at the heart of DOM-based On-Demand Javascript.

 function ensureUploadScriptIsLoaded() {
   if (self.uploadMessages) { // Already exists
     return;
   }
   var head = document.getElementsByTagName("head")[0];
   script = document.createElement('script');
   script.id = 'uploadScript';
   script.type = 'text/javascript';
   script.src = "upload.js";
   head.appendChild(script)
 }

The calling script just has to call this function. However, as mentioned in the Solution, it's downloaded asynchronously, so a check must be made here too.

 ensureUploadScriptIsLoaded();
 if (self.uploadMessages) { // If not loaded yet, wait for next sync
   uploadMessages(pendingMessages, onMessagesLoaded);
   ....

In most cases, the test will actually fail the first time, because the download function will return before the script's been downloaded. But in this particular application, that doesn't actually matter much - the whole synchronisation sequence is run every five seconds anyway. If the upload function isn't there yet, it should be there in five seconds, and that's fine for our purposes. If desired, we could be more aggressive - for instance, the script could be downloaded as soon as an edit begins.

XMLHttpRequest-Based On-Demand Javascript

The DOM manipulation technique is refactored here to use an XMLHttpRequest Call instead, where a server call retrieves upload.js and eval's it. As with the previous version, upload.js no longer appears in the initial HTML. Also, the invocation of ensureUploadScriptIsLoaded remains as before. The initial script - wiki.js differs only in the implementation of the Javascript retrieval. The text response is simply passed to eval.

 function ensureUploadScriptIsLoaded() {
   if (self.uploadMessages) { // Already exists
     return;
   }
   ** ajaxCaller.getPlainText("upload.js", function(jsText) { eval(jsText); }); **
 }

The actual Javascript must also change. If it simply defined a global function like before, i.e.:

 function(pendingMessages, messageResponseHandler) {
   ...
 }

the function would die when the evaluation completes. Instead, we can achieve the same affect by altering the browser window object (self).

 self.uploadMessages = function(pendingMessages, messageResponseHandler) {
   ...
 }

Related Patterns

HTML Response

The Javascript Response variant of this pattern is a companion to HTML Response and follows a similar server-centric philosophy in which it is the server-side that pushes instructions to the browser.

Predictive Fetch

Apply Predictive Fetch to On-Demand Javascript by downloading Javascript when you anticipate it will soon be required.

Browser-Side Cache

The nature of Javascript deployment means that a form of browser-side caching occurs naturally. That is, the results of calls are usually retained by virtue of the fact they automatically add new functions to the DOM.

Multi-Stage Download

On-Demand Javascript is a variant of Multi-Stage Download. The emphasis in Multi-Stage Download is on downloading semantic and display content rather than Javascripts In addition, that pattern is more about downloading according to a pre-scheduled sequence rather than downloading on demand.

Visual Metaphor

If you watch someone learning to use a new product, their experience will usually mimic On-Demand Javascript. They'll read the bare minimum, if anything, and get started playing with it. When they get stuck, they'll consult the instructions, but only enough to get over the next hurdle.

Want to Know More?

Dynamic data using the DOM and Remote Scripting A tutorial by Thomas Brattli.

Retrieved from "http://ajaxpatterns.org/On-Demand_Javascript"

 will load and display an element on the webpage in the given DIV element. twoBirds includes a modular site structure based on module directories.
-Since a twoBirds web-object may include other twoBirds web objects, the site overall complexity level in unlimited, still allowing quality management of the code.
+Since a twoBirds web-object may include other twoBirds web objects, the site overall complexity level is unlimited, still allowing quality management of the code.
 twoBirds is not yet out in the open, see [http://system.dlv.phpb002.de] for an early prototype. Contact the programmer if you want to get a copy of the current stage of development.
 <!-- -------------------------------------------------------------------- -->
 = Refactoring Illustration =

On-Demand Javascript

From Ajax Patterns

Revision as of 11:53, 6 December 2006

Contents

In A Blink

Technical Story

Problem

Forces

Solution

Decisions

How will you break modules down?

How will the script determine which Javascript to download?

At what stage will the script download the Javascript?

Real-World Examples

MapBuilder

Dojo Package System

JSAN Import System

twoBirds full AJAX

Refactoring Illustration

Introducing On-Demand Javascript to the Wiki Demo

Separate Javascript: Extracting `upload.js`

DOM-Based On-Demand Javascript

XMLHttpRequest-Based On-Demand Javascript

Related Patterns

HTML Response

Predictive Fetch

Browser-Side Cache

Multi-Stage Download

Visual Metaphor

Want to Know More?

Views

Personal tools

On-Demand Javascript

From Ajax Patterns

Revision as of 11:53, 6 December 2006

Contents

In A Blink

Technical Story

Problem

Forces

Solution

Decisions

How will you break modules down?

How will the script determine which Javascript to download?

At what stage will the script download the Javascript?

Real-World Examples

MapBuilder

Dojo Package System

JSAN Import System

twoBirds full AJAX

Refactoring Illustration

Introducing On-Demand Javascript to the Wiki Demo

Separate Javascript: Extracting upload.js

DOM-Based On-Demand Javascript

XMLHttpRequest-Based On-Demand Javascript

Related Patterns

HTML Response

Predictive Fetch

Browser-Side Cache

Multi-Stage Download

Visual Metaphor

Want to Know More?

Views

Personal tools

Separate Javascript: Extracting `upload.js`