System Test - Ajax Patterns

System Test

From Ajax Patterns

对于红彤美容整形也许大家还不太相信它的神奇,看看韩国好多整过形的美女演员出演的电视剧,整形前是多么地

难看,整形后判若两人,大家看到的都是他们整形后的美丽容颜,那在中国的整形美容行业发展的怎么样呢?我们

现在来设想下,如果要是把网络红人“芙蓉姐姐”整形成香港明星李嘉欣,那么结果会怎么样呢?她们两个不同风格的人能整形出来什么效果

?芙蓉能整出李嘉欣的形象气质出来吗? 美容整形在国外一直是很热门的产业,许多面部有缺陷的女孩子经过整形以后能成为耀眼的明星。一些胸部整形

的女星也能跟美国的艳星乔丹相媲美,演艺事业也蒸蒸日上,导演也亲睐她们,经常给她们抛来橄榄枝。[http://www.100woman.cn 处女膜修

复]手术,还你一个做纯洁女人的机会,双眼皮 手术,能给那些单眼皮女生一次跨越的机会,一下子跻身双眼皮行

列,男生们好象比较流行单眼皮,像Rain!割个双眼皮,轻松;做个双眼皮,方便;光子脱毛,快捷;

清除腋臭,简单;去整形美容医院,给自己一次完美的机会,去雅靓看看,能给你一个意外的惊喜! Evidence: 1/3

Tags: Automated FunctionalTest Javascript PerformanceTest Regression Robot UnitTest TDD Test


Contents

Ajax/Javascript Programming and Usability in "Ajax Design Patterns" Book

Goal Story

Devi has completed the first cut of a new Ajaxian application and wants to build an automated test of the whole thing. With the help of a testing application, she records a typical usage so it can be played back later. The software takes note of the final state, which will be used for verification during automated playback.


Problem

How can you test an entire Ajax application?


Solution

Build automated tests to simulate user behaviour and verify the results. A system test is one which exercises a complete system in much the same way as a user would. The most common category of system test is a functional test - verifying the functionality - but system testing also encompass qualities such as performance and robustness. While they are often conducted manually, with users clicking and mousing around, it's highly preferable to automate as much as possible in order to promote continuous integration.

Automated system tests remain somewhat difficult, but tools are improving. They can be broadly categorised as follows:

  • Browser Controller. A specialised "robot"-style testing tool will fire up the browser, then use the underlying operating system's API to create the effect of a user interacting with the browser. Several commercial controllers are able to record user actions for later playback. (caution: capture/replay is the less cost effective way of test automation)
  • Embedded Browser Script. Run the application in a standard browser, and embed a Browser-Side Test on the page. The test can use some Javascript trickery to simulate user activity. Don't let the browser aspect put you off, because it's feasible to have a continuous integration tool like Ant control coordinate Browser-Side Tests, as that pattern points out.
  • Browser Simulation. The application is executed by a model of a real browser. Typically, the simulation pulls a page from a URL somewhere and builds up a DOM to represent it. The DOM can then be manipulated, e.g. the simulation allows a button click operation to occur or a form to be submitted. While several frameworks like this exist, the important issue for Ajax developers is how well they simulate Javascript. Javascript interpreters are implemented in several languages, so it's possible for Browser Simulations to incorporate them in order to simulate a realistic Ajax interaction.

TODO Diagrams

The first two approaches involve a real browser, which is a more realistic test, but it does imply that the test must run on a machine capable of launching the browser, or somehow controlling it remotely and this may block the machine during test execution.

System tests require not just automation of user behaviour, but of result verification. As with other tests, results can be verified with a combination of assertions against the DOM and assertions against server-side state.


Tool Support

SWEA (Webius Software)

SWEA automates regression and functional testing for Web applications. The program creates an automation API for any Web application developed with HTML, DHTML or AJAX. The Web application becomes programmatically accessible from any .NET language. SW Explorer Automation Visual Designer records, replays test scripts and generates C# or VB.NET script code which can be used in NUnit or Visual Studio unit tests.

  • Commercial from Webius.

InCisif.net (InCisif Software)

InCisif.net is an automation tool designed to implement client-side functional testing of web applications under Internet Explorer 6.x or 7.x, using the C#, VB.NET or IronPython language with Visual Studio 2003, 2005 or express editions.

Commercial from InCisif Software

ItsNat, Natural AJAX (Innowhere Software)

ItsNat is a Component based Java AJAX Web Framework with built-in functional web testing support. ItsNat simulates a Universal W3C Java Browser in the server, the client DOM tree is basically a clone of the server W3C DOM tree with automatic synchronization from server to client.

The server can fire W3C DOM events and send them to the browser simulating user actions, these actions are usually processed by the server again as in a normal AJAX application. As the Java test code is in the server too, this test code can check the results as GUI changes (checking the expected changes in the server DOM tree) and/or the expected business behavior (checking changed/added/removed data).

  • Dual licensed: Affero GPL v3 / commercial license for closed source projects.

Squish/Web (froglogic)

Squish is an automated, functional GUI testing tool. It allows to record, edit and run web/ajax tests in different browsers (IE, Firefox, Safari, Konqueror, etc.) on different platforms without having to modify the test scripts. Supports different scripting languages for tests.

  • Commercial from froglogic

WebAii (artoftest)

WebAii is a .NET automation library provided for free to the quality assurance and developer community. It is built for Ajax and rich web application automation. Currently supports (IE,FireFox) (more browsers are planned). Plugs-in Visual Studio environment. Also supports a new approach to abstracting markup applications using TestRegions. Use with NUnit or Visual Studio Team Test.

  • Commercial from ArtOfTest, Inc. Provided Free of charge.

GhostTrain

Scriptaculous's GhostTrain is an Embedded Browser Script with recording capability. When you start the recording mode, a Scriptaculous test case begins to fill up inside the div. Each mouse and key action becomes a new line of the test case. Later on, you can add your own assertions about the application state. GhostTrain is a very promising work-in-progress.

Watir

Watir follows the Browser Controller approach, enabling a Ruby programmer to script IE behaviour and interrogate the DOM at any stage. You can easily integrate Watir commands into a standard test case. Example syntax:

  ie = Watir::IE.start("http://ajaxpatterns.org")
  ie.button(:name, "Search").click

WatiN

WatiN follows the Browser Controller approach based on Watir. WatiN enables a .Net programmer to automate web application testing with Internet Explorer. The framework can be used in any kind of tester application like NUnit, MBUnit, VSTS test, FIT, FitNesse or your own winforms or console app. Support for AJAX is included.

open source

Selenium

Selenium is an open-source tool by Thoughtworks developers, an Embedded Browser Script framework that works in IE, Mozilla, Firefox, and Safari. A Javascript Selenium component is embedded inside the browser application under test, and it communicates with a server-side component. Interesting, there's flexibility on the location of the server-side component. It can be placed in the web server, to provide a quick summary in the web page being developed. Alternatively, it can be run as a separate server, where the server controls the browser (like a Browser Controller would), and the embedded browser component merely transmits server commands to the application.

HTTPUnit

HttpUnit is an open-source Java testing framework, built on JUnit, and specifically designed for testing web applications. It works as a Browser Simulator, converting an HTML page into a Java HTMLPage object which wraps a Java DOM model of the original HTML. HttpUnit simulates Javascript by delegating to the Rhino Javascript engine, although support is limited and the project seems to be asleep.

DojoToolkit

DOJO Toolkit is a JavaScript toolkit. It comes with a UnitTest Module that uses Rhino to let you run tests as a Browser Simulator from an Ant task. Given the project's momentum and focus on Ajax, it's likely to evolve into a very solid product.

HtmlUnit

HmltUnit is an open-source Java library that can be considered as "browser for Java programs". It works like a browser that you can drive from your java code to get pages, fill form, click link, ... HtmlUnit is able to simulate Javascript execution like Internet Explorer or like Firefox (according to the chosen configuration). Simulation is already quite good and AJAX is fully supported.

Canoo WebTest

Canoo WebTest is an open-source (Apache license) tool to test web applications. Tests are specified (and not programmed) as Ant scripts and executed using HtmlUnit as underlying browser (therefore no display is needed). It provides high level abstraction allowing to write short, easy to maintain tests. Contained steps are not limited to HTML pages but allow to test xml, pdf, and Excel documents as well as Applets or Emails. Additionally the whole power of Ant can be used and it can be easily extended for application specific needs. Because WebTest is based on Ant it integrates directly in continuous integration tools like Cruise Control. Particularity of WebTest are the generated reports. Unlike other tools that just produce "JUnit like" reports (which are most of the time not enough to find a failure cause), WebTest results allow to follow the whole test execution and contain all received answers from the server(s).


Refactoring Illustration

The Ajax Patterns Reader is refactored here to include Scriptaculous's GhostTrain, discussed above. You can hit Escape anywhere in the document to open it up, and then use the buttons to record a test case, which can then be played back.

To hook up GhostTrain, the scriptaculous library files must be included (quite a lot to download, but keep in mind they don't need to be included in production):

  <script type="text/javascript" src="/run/Lib/js/sc/prototype.js"></script>
  <script type="text/javascript" src="/run/Lib/js/sc/controls.js"></script>
  <script type="text/javascript" src="/run/Lib/js/sc/effects.js"></script>
  <script type="text/javascript" src="unittest.js"></script>
  <script type="text/javascript" src="ghosttrain.js"></script>

The only other requirement is to register the GhostTrain object as a keypress handler. This also serves to start up GhostTrain if it's not yet running.

  window.onload = function() {
    Event.observe(document, 'keypress', GhostTrain.intercept, true);
    ...
  }


Related Patterns

Browser-Side Test

System Test frameworks that work as Embedded Browser Scripts are a form of Browser-Side Test.

Simulation Service

For integration tests, you can configure the server to run against a Simulation Service. That way, you'll have tight control over server behaviour, ensuring the final state is deterministic.


Want to Know More?

A Summary of Functional Testing Frameworks


Acknowledgements

Thanks to Scriptaculous developers, Jon Tirsen and Thomas Fuchs, for clarifying some issues on GhostTrain and their general testing framework. 热转印机 有机玻璃 IBM服务器 Dell服务器 IBM服务器 HP服务器 CISCO交换机 IBM服务器 顺义旅游 [1] 调查 单片机培训 工控机 北京搬家 北京搬家 会议服务 装饰装潢 展览制作 北京装潢公司 北京装饰公司 北京装修公司 北京月嫂 门窗厂 顺义 代开发票  北京物流 顺义房产  北京搬家 门窗厂 北京保洁    [2] 顺义 代开发票 北京汽车陪练  空调维修 空调回收 大金中央空调 电地暖 冷库 格力中央空调 约克中央空调 风幕机 劳保用品  北京物流公司 机柜 机柜 电机修理 净化工程 五粮液酒 燃气灶维修