[Plone-testing-team] Future of robot testing in Plone & add-ons
gotcha at bubblenet.be
Mon Oct 28 18:50:10 UTC 2013
Le 24/10/13 21:51, Asko Soukka a écrit :
> I'm currently (slowly) working on p.a.robotframework 0.7.0. For the
> revised docs, I'd love to have your opinion on, what should be the
> recommended use cases for Robot testing with Plone and add-ons?
Short answer: cross-browser testing of JS based features.
> Originally Robot Framework and its SeleniumLibrary were introduced to us
> for testing Java Script -based features in Plone. But Plone 5 all Java
> Script is be tested already in Plone-mockup
> and Selenium-tests would only make sense only for acceptance tests.
As far as I understand Plone-mockup is mainly unit-tested. Is that
correct ? Are the tests run in various browsers ?
For me, the goal of robot-framework usage has always been to do
acceptance testing of JS based features. And to build test suites that
would be runnable against as many browsers as possible.
The reason I had that goal is the following : when we did KSS, its JS
code was already fairly well unit-tested. However, it became quickly
obvious that this was far from enough... and that integration/acceptance
testing was also more than needed : the UI integration would break
randomly in new browsers or would break because of slight changes in html.
I understand that recent browsers are much more stable than the one we
were using back in 2006.
I still think that those acceptance tests are really needed.
Selenium tests can be run against all major browsers. Especially now
that Saucelabs allow us to run freely open source tests.
> On the other hand, Robot is easy and fun to learn and use. Yet, Selenium
> tests are still slow to run and pain to debug.
> p.a.robotframework 0.7.0 (with robotframework 1.8.2) includes helpers
> (and still in-progress docs) for creating Robot testable documentation
> with Robot (Selenium) generated screenshots. Practically all the stuff,
> I was piloting in collective.usermanual:
>> From CI point-of-view, Robot tests in documentation makes sense only
> with pure Robot Framework, without zope.testrunner (or robotsuite). They
> are not yet as optimal in layer setup/teardowns, but allow bundling
> tests so that there are less setup/teardown cycles than with
> What do you think, should I update docs to recommend using Robot only in
> documentation and direct to Timo's testing documentation for other tests?
As stated above, I think acceptance tests are indispensible.
> Also, if we can have acceptance testable documentation, are there need
> for acceptance outside the documentation?
Experience with doctests has shown that testable documentation is very
different from tests. It is very useful. But not enough. And often, do
not cover all use cases/corner cases.
> Any thoughts?
I'd be happy to be kept cced if there is feedback from the list.
PS: Asko, I was really happy that you received the Plone award. It is
well deserved... and I am happy to have a chance to congratulate you.
Godefroid Chapelle (aka __gotcha) http://bubblenet.be
More information about the Plone-testing-team