AjaxPatterns
Automated, FunctionalTest, Javascript, PerformanceTest, Regression, Robot, UnitTest, TDD, Test
Devi has completed the first cut of a new Ajax App and wants to build an automated test of the whole thing. With the help of a testing application, she records a typical usage so it can be played back later. The software takes note of the final state, which will be used for verification during automated playback.
Build automated tests to simulate user behaviour and verify the results. A system test is one which exercises a complete system in much the same way as a user would. The most common category of system test is a functional test - verifying the functionality - but system testing also encompass qualities such as performance and robustness. While they are often conducted manually, with users clicking and mousing around, it's highly preferable to automate as much as possible in order to promote continuous integration.
Automated system tests remain somewhat difficult, but tools are improving. They can be broadly categorised as follows:
A specialised "robot"-style testing tool will fire up the browser, then use the underlying operating system's API to create the effect of a user interacting with the browser. Several commercial controllers are able to record user actions for later playback.
Run the application in a standard browser, and embed a “Browser-Side Test” on the page. The test can use some Javascript trickery to simulate user activity. Don't let the browser aspect put you off, because it's feasible to have a continuous integration tool like Ant control coordinate “Browser-Side Test”s, as that pattern points out.
The application is executed by a model of a real browser. Typically, the simulation pulls a page from a URL somewhere and builds up a DOM to represent it. The DOM can then be manipulated, e.g. the simulation allows a button click operation to occur or a form to be submitted. While several frameworks like this exist, the important issue for Ajax developers is how well they simulate Javascript. Javascript interpreters are implemented in several languages, so it's possible for Browser Simulations to incorporate them in order to simulate a realistic Ajax interaction.
The first two approaches involve a real browser, which is a more realistic test, but it does imply that the test must run on a machine capable of launching the browser, or somehow controlling it remotely.
System tests require not just automation of user behaviour, but of result verification. As with other tests, results can be verified with a combination of assertions against the DOM and assertions against server-side state.
Scriptaculous's GhostTrain is an Embedded Browser Script with recording capability. When you start the recording mode, a Scriptaculous test case begins to fill up inside the div. Each mouse and key action becomes a new line of the test case. Later on, you can add your own assertions about the application state. GhostTrain is a very promising work-in-progress.
Watir follows the Browser Controller approach, enabling a Ruby programmer to script browser behaviour and interrogate the DOM at any stage. You can easily integrate Watir commands into a standard test case. Example syntax:
ie = Watir::IE.start("http://ajaxpatterns.org")
ie.button(:name, "Search").click
Watir is IE-specific right now.
Selenium is an open-source tool by Thoughtworks developers, an Embedded Browser Script framework that works in IE, Mozilla, Firefox, and Safari. A Javascript Selenium component is embedded inside the browser application under test, and it communicates with a server-side component. Interesting, there's flexibility on the location of the server-side component. It can be placed in the web server, to provide a quick summary in the web page being developed. Alternatively, it can be run as a separate server, where the server controls the browser (like a Browser Controller would), and the embedded browser component merely transmits server commands to the application.
HttpUnit is an open-source Java testing framework, built on JUnit, and specifically designed for testing web apps. It works as a Browser Simulator, converting an HTML page into a Java HTMLPage object which wraps a Java DOM model of the original HTML. HttpUnit simulates Javascript by delegating to the Rhino Javascript engine, although support is incomplete.
DOJO Toolkit is a Java-based Browser Simulator that uses Rhino to let you run tests from an Ant task. given the project's momentum and focus on Ajax, it's likely to evolve into a very solid product.
The Ajax Patterns Reader is refactored here to include Scriptaculous's GhostTrain, discussed above. You can hit Escape anywhere in the document to open it up, then use the buttons to record a test case, which can then be played back (Figure 1.114, “Ghost Train”).
To hook up GhostTrain, the scriptaculous library files must be included (quite a lot to download, but keep in mind they don't need to be included in production):
<script type="text/javascript" src="/run/Lib/js/sc/prototype.js"></script>
<script type="text/javascript" src="/run/Lib/js/sc/controls.js"></script>
<script type="text/javascript" src="/run/Lib/js/sc/effects.js"></script>
<script type="text/javascript" src="unittest.js"></script>
<script type="text/javascript" src="ghosttrain.js"></script>
The only other requirement is to register the GhostTrain object as a keypress handler. The handler will take care of any other initialisation:
window.onload = function() {
Event.observe(document, 'keypress', GhostTrain.intercept, true);
...
}
System Test frameworks that work as Embedded Browser Scripts are a form of “Browser-Side Test”.
For integration tests, you can configure the server to run against a “Simulation Service”. That way, you'll have tight control over server behaviour, ensuring the final state is deterministic.