How to run FlexUnit in a continuous build

February 1st, 2009 · 5 Comments ·

So you have a flash or Flex code with lots of unit-tests. That is great, but how do you get them running in a continuous build? Sure you can run the tests manually, but this only gives you a visual green or red bar, what you really want is a X-Unit XML file so that you can run the test in an automated fashion and display or email the result to everyone on your team. The problem is that flash runs in a sandbox and as a result the FlexUnit is not allow to tell anyone about the success or failure of any test, which makes publishing the results kind of hard.

Enter Adobe Air to the rescue! Adobe Air is a flash player, which is outside of the sandbox. As a result Adobe Air application can write to local file system. And because Adobe Air is flash it is 100% compatible with your flash / Flex code which you are trying to test. Here is what you need:

Basic idea is straight forward. Use an ant script to compile your ActionScript code together with unit tests into an Air executable. The executable runs the tests with special XMLListener which records the tests into X-Unit complaint XML file and writes it out to the filesystem. The Air application than quits with status code which equals to the number of test failures. The continuous build can than use the status code and the XML report to publish the test results.

For extra points…

The same problem (the results are stuck in browser) exists when testing JavaScript code. Lucky for us Adobe Air includes WebKit. This means that the same Air test runner can execute your JavaScript tests as well. Upon the completion of JavaScript tests the test runner can traverse the HTML DOM and collect the JavaScript results and publish them in the same way.

– Enjoy…

Tags: Uncategorized

5 responses so far ↓

  • Luke Bayes // Feb 2, 2009 at 12:24 pm

    There are still some problems here.1) It’s ideal to run your test harness in an environment that is as close as possible to your production environment, and unless your application is intended for AIR, then compiling to AIR includes a lot of unintended code. The same goes for ActionScript only projects that are tested using FlexUnit, Flex applications are not the same as ActionScript applications.2) Just like the desktop and browser debug Flash Players,  AIR launches a GUI dialog with any uncaught runtime exceptions. This means that your CI machine will hang and at best – simply fail with a timeout and no useful information, whenever an exception occurs with the frame clock at the top of the stack.The solution?<a href=””>AsUnit</a> and <a href=””>ProjectSprouts</a>.Among many things, Sprouts includes a Continuous Integration Rake task that runs in the Flex Debugger instead of the Flash Player, this gives us the ability to capture uncaught runtime exceptions, output the failure stack trace and all variables (and values) in the local scope where the failure occurred, and close down properly.Of course test results, including failures, are always output to an XUnit XML results file.Check it out and let us know what you think!Thanks

  • misko // Feb 2, 2009 at 12:48 pm

    I was not aware of those projects, let me check them out…

  • zwetan // Feb 3, 2009 at 3:59 am

    Hi, I took another shot at the problemthe Flash Player or AIR runtime both use the Tamarin VM to execute ActionScript, and on top of it add their respective APIso basically I added few extras to the Tamarin VM and ported my own unit tests framework to run in a CLI, so my unit tests can run in that VM on the command-linea longer more detailled explanation can be found here let me know what do you think about it :)

  • jensendarren // Apr 22, 2010 at 5:42 am

    Thankyou for your informative insight. I am very new to Flex and have been literally thrown into an existing project which already has a set of tests written by another developer. My task is to get these tests to run as part of our nightly build.

    I took the example you provided and applied it against our own test runner using Rake instead of ANT. After half a day of figuring out how Flex works and how to compile it etc, etc, I adjusted our runner to write out an XML file and successfully compiled it using the AIR libraries.

    However, when I run this on our deployment server (Ubuntu 9.04 Server) I see the following error “Gtk-WARNING **: cannot open display”. How can I get adl to run without a GUI?

  • misko // Apr 22, 2010 at 5:14 pm

    have you tried XVFB?

Leave a Comment