Update on 1.2 testing

Colin Clark colinbdclark at gmail.com
Wed Apr 14 15:53:20 UTC 2010


Hi Michelle,

On 2010-04-14, at 10:30 AM, Michelle D'Souza wrote:
>> Would it not be sufficient to write a unit test that shows the bug without the patch and passes once the patch had been applied?
> 
> Interestingly it was actually an existing test that caught the bug. Which is good but points to a hole in our infrastructure. We depend on people to run the automated tests instead of the tests being run upon commit or during the nightly build. We've known that this is an issue for a long time but never quite found the time to rectify it. So this time around *4* of us - the bug fixer, the two independent reviewers (I'm one of them) and the QA tester didn't run the unit tests. And all of us like unit tests and generally run them - but we were crunched and rushed so it slipped through. Luckily that final testing did catch the slip and the test is passing again with the patch Justin created.


Fair enough. Being one of those four, I thoroughly apologize for not catching the failing test. I'm sure we're all feeling a little embarrassed about it, and I guess this can serve as a good reminder to us about testing rigour.

So, to the question of test automation, there are some options:

1. The TestSwarm server is now fairly easily installable. It's a set of PHP scripts and a bit of Apache configuration. We'd each probably have to consistently keep a browser window or two open to the TestSwarm server, but it should allow us to automate the tests in a fairly elegant and distributed way.

2. The Opencast community has done some work with JavaScript test automation using Maven. They're using a tool called JSTestDriver, which does include some support for QUnit tests. It is reminiscent of the distributed features in JSUnit, where there's a server running on each platform receiving results from the test runner, and then distributing it back to the machine that initiated the the test run.

http://code.google.com/p/js-test-driver/

Thoughts?

Colin

---
Colin Clark
Technical Lead, Fluid Project
http://fluidproject.org




More information about the fluid-work mailing list