Distributed Events - Ajax Patterns

Distributed Events

From Ajax Patterns

Evidence: 1/3

Tags: Events Feed Listener Messaging MVC Notification Observer PublishSubscribe RSS Refresh Semantic Synchronise Synchronize Update WebFeed


Contents

In A Blink

Events back and forth


Technical Story

Devi's producing a web application for auditors at a financial institution, aiming to highlight substantial transactions. Relevant transactions are already published on the enterprise messaging system, and Devi transforms it into an RSS feed on the web server. The Ajaxian browser application checks the feed every few seconds, and updates the view whenever it detects a substantial transaction occurred.


Problem

How do you decouple code in a complex application?


Forces

  • Ajaxian applications involve at least two tiers: a browser tier and a web server tier. In practice, the web server tier is often dependent on further tiers and external systems.
  • Each tier can be quite complicated, containing many stateful entities (objects, HTML controls, regular variables).
  • The state of all these entities must often be synchronised, in order to keep users and external systems up-to-date. The synchronisation needs to occur within a tier as well as across tiers - for example an HTML table needs to change whenever browser-side user preferences change, but also whenever the server-side database changes.
  • Keeping all these objects synchronised can become complex - there are n(n-1) possible message paths from one object to another.


Solution

Keep objects synchronised with an event mechanism. This is a classic software pattern applied to Ajax, related to the "Observer" (Gamma et al.) and "Publisher-Subscribe" (Buschmann et al.) patterns, and also a key feature in the classic "Model-View-Controller" architecture. DHTML already provides a mechanism for low-level events, but the events discussed here are more semantic in nature, i.e. related to business and application concepts such as "account deleted". Note that the term "event" is used in a broad manner, to mean any subscription-based approach that alleviates the need for direct calls from source to destination. Any publish-subscribe messaging mechanism falls under this definition.

Here's a scenario to motivate the concept of events. Say you have 10 objects with inter-dependent states. That is, when one object changes, any number of the other 9 must change accordingly. The naieve implementation would endow each object with an understanding of the other 9 objects. Each time its changed, an object would then tell each other object how to update. Each object now knows the other 9 intimately. When one changes, the other 9 must be updated - a major blow to encapsulation.

As with many programming problems, you can create a better solution by adding another layer of indirection. In this, an event mechanism. Let each object broadcast changes instead of directly telling others how to respond. The changes should generally occur in semantic terms - rather than saying "someone's clicked me", an object should say "counting module has begun". And let any object register to be notified whenever a message like this occurs. For larger systems, thinking in terms of events is easier as it breaks down the synchronisation logic. You have one simple task to make objects broadcast events whenever they occur. And you have a separate task to decide how objects should actually respond to events, if they care about them at all.

On the web, this pattern can be applied in different ways:

  • Server-to-Browser: Page elements can be kept in sync with server objects.
  • Browser-to-Browser: Page elements can be kept in sync with each other.
  • Browser-to-Server: Server objects can be kept in sync with browser objects.
  • Server-to-Server: Server objects can be kept in sync with each other.

Browser-to-Server and Server-to-Server are beyond the scope of the Ajax Patterns because they are more about server-side architecture.

Server-to-Browser propagation works something like this. There's a Server Event Manager, with two responsibilities. First, it runs a Periodic Refresh to check if any server activity has occurred. Second, it offers an event propagation facility so that interested browser entities can register to discover any changes. When a server change occurs, the Server Event Manager constructs an event object based on the server's output, and passes it any interested listener. To do this, the manager must retain a collection of listeners and the event types they are listening to. The minimal set of services would then be:

  • addListener(eventType, listener);
  • removeListener(eventType, listener);

These listeners can be callback functions, like the callback function used by XMLHttpRequest. Or, they can be objects which contain a function with a standard callback name for the event type being registered, such as onUpdate().

With Browser-to-Browser propagation, you can also have a central server manager to accept events and notifications. Alternatively, each object can be responsible for creating and propagating events specific to itself. That is, each object capable of generating events needs to allow other objects to register themselves for the events.

Observer is a special case of this pattern, one which appears frequently. The events concern not user actions, but change notifications. Event listeners are observing an object and responding to its state. Often, this happens to keep state in sync. An HTML table, for example, can render the latest state of a 2-dimensional array on the server-side.

Finally, note that this pattern is somewhat speculative and open-ended, but the main point should be clear: to add a layer of intermediation so that objects can encapsulate their own responses to system activity, rather than being told what to do by other objects.


Decisions

Will you publish a history or just the current state?

Given the nature of HTTP, server data must be retrieved periodically - the server can't directly call the browser when something happens. If the server only exposes the current state, there are two problems:

  • Change detection: The browser can only infer a change has occurred by comparing the previous value, or perhaps by using a version ID or timestamp.
  • Intermediate states: The browser will miss intermittent values, i.e. those which began

TODO temporal diagram

Neither of these are showstoppers, so exposing only the current state may well be feasible. The alternative is to publish a history of states along with timestamps. That's more work to output and parse, and it also requires some form of storage on the server. If you are following that option, it's worth formatting the changes using a standard feed-based approach such as RSS or Atom. You'll benefit from the abundance of libraries for both browser and server. As a bonus, the service will be generic enough to be used by external clients, if that's desired.

For Observer-style events, Will you propagate the details of what's changed, or just point to the object that's changed?

Often, events concern an object that's changed state. Sometimes, you only need to propagate a pointer to the object; the recipient will then interrogate the object as required. Other times, you need to send the change itself. The latter approach is useful for functionality which requires not just the object's current state, but the nature of the change. For example, imagine you have an auditing function which logs whenever a budget balance has been increased. It will be much easier if the change event indicates that an increase has occurred. Otherwise, it will have to manually compare the budget against its previous state. For server-to-browser events, indicating the change may be better for performance since it may alleviate a follow-up call to the server.

What information will accompany the event notification?

When an event occurs, there's usually several pieces of information to pass to listeners. For example: a source object (an object that's just changed), the nature of the change or action that's occurred, meta-information such as event time and unique ID. You can pass this information in different ways:

  • String Message: Pass the information as a single string.
  • Single Event object: Pass in a single Event object containing attributes for each piece of information.
  • Parameter list: Pass the information as separate parameters.

Each style has its strengths. A string message is the most flexible approach and has the benefit of being a portable format which will work on the server as well. Unfortunately, a string message must often be parsed and formatted to convert to and from useful Javascript values. A single event object is easier to manipulate and, like a string message, can usually be extended without breaking existing code - the callback function still takes a single value. You can create a factory function to create the event and expose properties, so its interface can be made explicit if you so desire. Finally, a parameter list makes for a cleaner callback function implementation, since you don't have to extract variables from a wrapper object. However, a long list of parameters is cumbersome and difficult to maintain.


Real-World Examples


Code Examples

ActiveMQ

Apache ActiveMQ is an open-source implementation of Java Messaging Service (JMS), an official Sun standard for enterprise messaging. As such, it provides a way to pass Java objects and strings between different proceses, and includes a publish-subscribe mechanism allowing a process to subscribe for all messages in a particular "topic".

Normally, the processes run server-side, but using a servlet adaptor, ActiveMQ effectively gives the web application, through Javascript, full ability to send and receive messages.

MapBuilder

MapBuilder is a framework for mapping websites, heavily influenced by MVC. The model holds application data such as current maps, positions, and dimensions. The configuration process wires each model to a number of interested widgets, all of which receive notifications when the model has changed.

Dojo Events

Dojo is a comprehensive framework aiming to simplify Javascript development. One thing Dojo does is enhance Javascript's standard event management. This includes publish-subscribe functionality. You register one or more functions as publishers and one or more functions as subscribers. When a publisher function is called, all of the subscriber functions will also be called.


Refactoring Illustration

The Basic Wiki Demo has a single callback function which serves two purposes: to parse the incoming message and to display it to the user. That's okay for a simple application, but what if we wanted to scale up the display operation: displaying different messages in different ways, or performing some action when a single message has changed. It won't be a great surprise that Distributed Events are one way to make the browser script more modular, and this refactoring shows how.

Refactoring to an Event Mechanism

The first refactoring lays the groundwork by introducing for a richer message handling by introducing an event mechanism. There are some minor user-interface differences, for coding convenience: (a) instead of a single "synchronise" point, downloading and uploading are split into independent timing mechanisms; (b) there's no more background colour change while waiting for a message to upload; (c) a One-Second Spotlight effect now occurs when a message has updated, to compensate for the loss of colour change. Also, note that a different version of ajaxCaller is used, one which allows a callback object to be specified in addition to a callback function.

A model object has been introduced to track the state of each message and to play the role of an event manager, notifying listeners of changes. One type of listener receives notification of any new messages. The other type receives notification of updates to a specific message. New message listeners are held in a single array. Update listeners are held in an array of arrays, with all subscribers to a particular message held in an array that's keyed on the message ID.

 newMessageListeners: new Array(),
 messageUpdateListenersById: new Array(),

 addNewMessageListener: function(listener) {
   this.newMessageListeners.push(listener);
 },
 addMessageUpdateListener: function(messageId, listener) {
   var listeners = this.messageUpdateListenersById[messageId];
   listeners.push(listener);
 },

Notification then works by iterating through the collection of relevant listeners.

 notifyNewMessageListeners: function(newMessage) {
     for (i=0; i<this.newMessageListeners.length; i++) {
       this.newMessageListeners[i](newMessage);
     }
 },
 notifyMessageUpdateListeners: function(updatedMessage) {
   var listenersToThisMessage =
     this.messageUpdateListenersById[updatedMessage.id];
   for (i=0; i<listenersToThisMessage.length; i++) {
     listenersToThisMessage[i](updatedMessage);
   }
 }

How do these events arise? The model object must be started manually, and will then periodically poll the server.

 start: function() {
   this.requestMessages();
   setInterval(this.requestMessages, DOWNLOAD_INTERVAL);
 }
 ...
 requestMessages: function() {
   ajaxCaller.getXML("content.php?messages", messageModel.onMessagesLoaded);
 },

As before, the server provides an XML Message describing all messages. The model steps through each message in the XML, constructing an equivalent Javascript object. If the message ID is unknown, the new message listeners are notified, and an array of update listeners is also created for this new message. If the message differs from the current message with the same ID, it's changed, so all the update listeners are notified. Recall that the message update notification is fine-grained: only listeners to a particular message ID are notified, hence the extraction of a message-specific list of listeners.

 onMessagesLoaded: function(xml, callingContext) {
   var incomingMessages = xml.getElementsByTagName("message");
   for (var messageCount=0; messageCount<incomingMessages.length;
        messageCount++) {
     var messageNode = incomingMessages[messageCount];
     var content = this.getChildValue(messageNode, "content");
     content = (content==null ? "" : unescape(content));
     var incomingMessage = {
       id: this.getChildValue(messageNode, "id"),
       lastAuthor: this.getChildValue(messageNode, "lastAuthor"),
       ranking: this.getChildValue(messageNode, "ranking"),
       content: content
     };
     var currentMessage = this.messagesById[incomingMessage.id];
     if (!currentMessage) {
       this.messageUpdateListenersById[incomingMessage.id]=new Array();
       this.notifyNewMessageListeners(incomingMessage);
     } else if (!this.messagesEqual(incomingMessage, currentMessage)) {
       this.notifyMessageUpdateListeners(incomingMessage);
     }
     this.messagesById[incomingMessage.id] = incomingMessage;
   }
 },
 getChildValue: function(parentNode, childName) {
   var childNode = parentNode.getElementsByTagName(childName)[0];
   return childNode.firstChild == null ? null : childNode.firstChild.nodeValue;
 },
 messagesEqual: function(message1, message2) {
   return    message1.lastAuthor == message2.lastAuthor
          && message1.ranking == message2.ranking
          && message1.content == message2.content;
 }

A new messagesDiv object has also been created to encapsulate the message handling logic. On startup, it subscribes for notification of new messages. For each message, it performs a similar function to what was previously done on each update: it creates all the message information and appends to the page, along with a newly-introduced visual effect (courtesy of Scriptaculous).

 start: function() {
   messageModel.addNewMessageListener(this.onNewMessage);
 },
 ...
 onNewMessage: function(message) {
   var messageArea = document.createElement("textarea");
   messageArea.className = "messageArea";
   messageArea.id = message.id;
   messageArea.serverMessage = message;
   ...
   messageDiv.appendChild(lastAuthor);
   messageDiv.appendChild(messageArea);
   ...
   $("messages").appendChild(messageDiv);
   Effect.Appear(messageDiv);
   ...
 }

The messageDiv has another responsibility: it must update the display when a message has updated. Thus, it registers itself as a listener on each message. The easiest way to do this is upon adding each new message.

 onNewMessage: function(message) {
   ...
   messageModel.addMessageUpdateListener(message.id, function(message) {
     var messageDiv = $("messageDiv" + message.id);
     var lastAuthor = messageDiv.childNodes[0];
     var messageArea = messageDiv.childNodes[1];
     if (messageArea.hasFocus) {
       return;
     }
     lastAuthor.innerHTML = message.id + "."
       + "" + message.lastAuthor + ""+"."
       + message.ranking;
     messageArea.value = message.content;
     Effect.Appear(messageDiv);
   });
 },

Compared to the previous version, we're now only redrawing a message when it's actually changed. Using an event mechanism has helped to separate the logic out. Now, it's the messageDiv itself that decides how it will look after a message comes in, which is much more sane than the message-receiving callback making that decision.

Introducing a watchlist

This refactoring wouldn't be very useful if we stopped with an event mechanism. Good for your resume perhaps, but we haven't yet added any functionality to justify the effort; it's basically the same application as before. Not to worry: Watchlist Wiki Demo to the rescue. A new watchlist monitors interesting messages, so that when a message you're watching is updated (by you or someone else), the watchlist will add a summary line.

To start with, the HTML now includes a watchlist table.

  <div id="summary">
    <table id="watchlist">
      <tr>
        <th>Author</th>
        <th>Message</th>
      </tr>
      <tbody id="watchlistBody"></tbody>
    </table>
  </div>

Which messages are in your watchlist? That's determined by a new checkbox control, one per message.

 onNewMessage: function(message) {
   ...
   var watching = document.createElement("input");
   watching.type = "checkbox";
   watching.messageId = message.id;
   watching.onclick = onWatchingToggled;
   ...
 }

When the user wants to watch a message, they select its checkbox. A single function updates the watchlist for all chosen messages. Remember that message update events are fine-grained, so we need to ensure this callback is registered to receive notifications for all the chosen messages, and nothing else. So when a user un-selects a message, we'll de-register the function as a listener on that message. Note that this functionality necessitated the creation of a de-registration function, something that was never required in the previous version.

 function onWatchingToggled(event) {
   event = event || window.event;
   var checkbox = event.target || event.srcElement;
   if (checkbox.checked) {
     messageModel.addMessageUpdateListener(checkbox.messageId, onWatchedMessageUpdate);
   } else {
     messageModel.removeMessageUpdateListener(checkbox.messageId, onWatchedMessageUpdate);
   }
 }

onWatchedMessageUpdate will now receive notification of any new messages that are being watched. It simply adds a summary row to the table and runs a visual effect.

 function onWatchedMessageUpdate(message) {

   var summary = message.content;
   if (summary.length > 35) {
     summary =   summary.substring(0, 15) + "..."
               + summary.substring(summary.length - 15);
   }

   var row = document.createElement("tr");

   var authorCol = document.createElement("td");
   authorCol.className = "authorSummary";
   authorCol.innerHTML = message.author;
   row.appendChild(authorCol);

   var contentCol = document.createElement("td");
   contentCol.className = "contentSummary";
   contentCol.innerHTML = summary;
   row.appendChild(contentCol);

   if ($("watchlistBody").childNodes.length > 0) {
     $("watchlistBody").insertBefore(row, $("watchlistBody").childNodes[0]);
   } else {
     $("watchlistBody").appendChild(row);
   }
   Effect.Appear(row);

 }

We now have two independent functions that receive notifications of new messages arriving from the server. Each can use the information however it pleases. This is a much more scaleable approach than having the server message recipient dictate how the browser should respond.


Alternatives


Related Patterns

Semantic Response

This pattern relies on the server deliver Semantic Responses, to make it easy for the event manager to comprehend server activity and state, and therefore facilitates propagation within the browser.

RESTful Service

Distributed Events usually involve a browser element observing a server-side entity. REST is ideal for this purpose as it provides a simple, standard, way to expose server state.

XML Data Island

If the server responds with XML and you need to retain state locally, e.g. to track differences, an XML Data Island would be useful.


Visual Metaphor

The traditional newspaper analogy still works. People can subscribe to any number of newspapers and each newspaper can have any number of subscribers. The algorithm does not explicitly mention any particular subscriber; rather, when a newspaper comes out, it simply loops through each subscriber and sends a copy to them.


Want to Know More?


Acknowledgements

Cameron Shorter, core MapBuilder developer, helped inspire this pattern by drawing attention to the MVC approach of MapBuilder.