[Tech] IE issue

Justin Obara obara.justin at gmail.com
Tue Dec 22 15:37:32 UTC 2009


Hello,

As Colin and Dan have mentioned, automated GUI testing is very brittle, especially when you factor in cross browser support. Below I've mentioned some problem areas, these are not all. It seems like a lot of the tools solve one of the problems while missing the other. This may be okay for your particular use case. It all depends on what you need to do, what your expectations for the tests are, and ensuring that the cost of maintenance and development don't out weigh the cost of manual testing. Also don't forget that you will always have to do at least some manual testing, for things like visual styling and etc. 

There are a couple of methods that I have seen to find an element on the screen. One is to use the xpath or something like that to point at an element, another is the pixel position. Here is a list of some issues with these.

xpath:

can be different in different browsers
will break if path changes due to design change

pixel position

can change with window size
can be different in different browsers even with same window size

Another issue that I encountered is that simulated events are not always passed along. Dojo got around this by using a java applet to call java's robot, which makes actual system calls. 

I'm not sure how far along you are in your setup, but here is what I would like to get setup first for Fluid. We have a suite of unit tests that we can manually trigger to run. I'd like to get TestSwarm setup and have a machine running with vm's for all of our configurations allowing us to easily have all of our unit tests running automatically.

Here are some links:

Doh Robot:
http://www.dojotoolkit.org/2008/08/11/doh-robot-automating-web-ui-unit-tests-real-user-events

TestSwarm:
http://testswarm.com/

Hope that's been helpful
Justin

On 2009-12-21, at 3:08 PM, Aron Roberts wrote:

> Thanks much, Dan and Colin, for these great summaries.  It'll be
> interesting to see Justin's perspectives, as well.
> 
> Dan:
>> I agree re the brittleness issue with my experience of selenium,
>> particularly on very dynamic interfaces. I had a project quite similar to
>> collectionspace where I was always rearranging things, and doing major UI
>> refactors. I spent most of my time updating the selenium selectors!
> 
> Colin:
>> Automated acceptance testing is a holy grail we've been questing after for about three years now. It'll be awesome when we can do it. Biggest problems: brittleness and "reach." Many scripted acceptance tests tend to come crashing down when design changes occur because they're tied to concrete features in the markup such IDs and the like. The reach issue is simply that it can be hard to automate some tasks that extend beyond the insides of the browser frame.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.idrc.ocad.ca/pipermail/fluid-work/attachments/20091222/bebff624/attachment.htm>


More information about the fluid-work mailing list