[Product-Developers] Re: Selenium tests

[Martin Aspeli]

Hi Balazs,


> For the recording of tests, Selenium IDE is used. For the running of 
> tests, Zelenium is used which means it runs on every browser. 
> It was not a goal to be runnable from within
> PloneTestCase, instead we wanted to make a group of tests 
> runnable from a browser with a single click. 
> 
> It also does not use DemoStorage but operates on an existing server, but 
> this could be extended if needed.

Then how do you do test setup and tear-down?

> For the test runner we decided not to use setup and teardown (like 
> PloneTestCase) instead we applied a simple testlayer based approach.
> The main reason is that running tests with Zelenium is slow and thus 
> separating all tests in the same layer with setup/teardown would be 
> unbearably slow in practice. However layers work effectively, although tests 
> are not allowed to leave trash behind, in order for this to work.

They probably will, though - I mean, a test that crates content will 
leave a content item in place. I don't think it's practical to make all 
tests reciprocal so that they also explicitly delete what they created etc.

> As far the results, the testrunner works very well in practice, I am 
> using it to test plone.app.kss on every browser before releasing. 
> There are not yet many tests written for Plone, but this is an issue separate 
> from the testrunner itself.
> 
> You can easily check out this method by using its buildout and following
> the step to step documentation.
> 
> 
> The second approach that Sylvain was working on the sprint is funittest 
> and it may also fit your needs:
> 
> This is a different method, it has a model based approach for creating 
> tests. Recording tests with Selenium IDE is not possible, you write code 
> only. For running the tests Selenium RC is needed which means additional 
> requirements for installation on the server side. May not work on all 
> platforms and browsers, but it has a working buildout available.
> 
> We are looking for a solution to allow a combination of both kinds of tests,
> for the Plone testing story. There has been a long discussion about this on
> the sprint, and we came to important conclusions.

I think the only complex bit is to ensure that you get:

  - set up based on layers
  - tear-down/roll-back after each test

Selenium will start the browser and do the automation, even if you're 
running the test in PloneTestCase.

Martin

-- 
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book