Update on 1.2 testing
colinbdclark at gmail.com
Wed Apr 14 15:53:20 UTC 2010
On 2010-04-14, at 10:30 AM, Michelle D'Souza wrote:
>> Would it not be sufficient to write a unit test that shows the bug without the patch and passes once the patch had been applied?
> Interestingly it was actually an existing test that caught the bug. Which is good but points to a hole in our infrastructure. We depend on people to run the automated tests instead of the tests being run upon commit or during the nightly build. We've known that this is an issue for a long time but never quite found the time to rectify it. So this time around *4* of us - the bug fixer, the two independent reviewers (I'm one of them) and the QA tester didn't run the unit tests. And all of us like unit tests and generally run them - but we were crunched and rushed so it slipped through. Luckily that final testing did catch the slip and the test is passing again with the patch Justin created.
Fair enough. Being one of those four, I thoroughly apologize for not catching the failing test. I'm sure we're all feeling a little embarrassed about it, and I guess this can serve as a good reminder to us about testing rigour.
So, to the question of test automation, there are some options:
1. The TestSwarm server is now fairly easily installable. It's a set of PHP scripts and a bit of Apache configuration. We'd each probably have to consistently keep a browser window or two open to the TestSwarm server, but it should allow us to automate the tests in a fairly elegant and distributed way.
Technical Lead, Fluid Project
More information about the fluid-work