[Plone-testing-team] Fwd: Re: Fwd: Re: JavaScript testing and integration

Timo Stollenwerk tisto at plone.org
Wed Jul 17 16:01:09 UTC 2013


-------- Original-Nachricht --------
Betreff: Re: Fwd: Re: [Plone-testing-team] JavaScript testing and
integration
Datum: Wed, 17 Jul 2013 17:10:25 +0200
Von: Rok Garbas <rok at garbas.si>
An: Timo Stollenwerk <tisto at plone.org>,

Hi from Bastille Indiana Sprint,

Quoting Timo Stollenwerk (2013-07-17 07:18:24)
> Hi Rok,
> 
> we are currently discussing about JS testing on the testing team
> mailinglist (plone-testing-team at lists.plone.org).
> 
> Would you be so kind to give us a short heads up about your plans so we
> can look into the options and evaluate how we can integrate those tests
> into our CI setup?
> 

in short javascript tests are run outside plone/zope/python world. to
run them
you'll have to do (you'll need to have git and npm installed):

    % cd mockup
    % make bootstrap
    % make test-ci

currently we didn't implement yet this test-ci since we're busy with getting
release out for plone.app.widgets/toolbar as soon as possible (should happen
this week for sure this time, since we're almost there). but we thought
about
that as well when we were picking out the tools to work with javascript.

i also commented inline below, so please FW it to others.


> -------- Original-Nachricht --------
> Betreff: Re: [Plone-testing-team] JavaScript testing and integration
> Datum: Tue, 16 Jul 2013 11:14:54 -0700
> Von: Ross Patterson <me at rpatterson.net>
> An: Asko Soukka <asko.soukka at iki.fi>
> Kopie (CC): plone-testing-team at lists.plone.org
> 
> On Fri, Jul 12, 2013 at 8:18 AM, Asko Soukka <asko.soukka at iki.fi> wrote:
> > Gil Forcada wrote:
> >>
> >> So I'm about to invest some time looking for how to make and integrate
> >> in plone.app.testing a JavaScript tests runner.
> >>
> >> What's the status quo right now? Any prior art on that? Rok is using
> >> Buster.JS right?
> >
> >
> > AFAIK, there is no status quo.
> >
> > As said in the ticket, Ross has made BusterJS integration for
> > zope.testrunner (https://github.com/plone/buster-selenium) so that JS tests
> > would be run and reported with other zope.testrunner tests.
> >
> > Rok, however, suggests that we should follow the best practices of JS
> > community and use JavaScript-based testrunner, especially not
> > zope.testrunner, for JavaScript unit tests.
> 
> These are not exclusive.  There's nothing in the zope.testrunner
> integration that keeps the buster.js tests from being run any other
> way.
> 

we've moved away from BusterJS because of many reasons:
 - inactive development (as of this day last commit happend 6 months ago)
 - poor OOTB integration with testing frameworks (Mocha,Jasmine,...)
 - poor (and buggy) reports for coverage
 - hacky support for AMD (RequireJS)


> > Rok has been advertising Mocha + Chai + TestEm combo. Mocha provides syntax
> > for test suites (and also a test runner), Chai provides assertions and
> > TestEm provides a continuous test runner, especially, for test writing
> > purposes. (For CI use, I'm not sure if TestEm has yet enough community to
> > stay alive.)
> 
> Do these provide for setup and tear down of the browser slave?  That's
> the primary value I was aiming for in buster-selenium, it re-uses
> selenium tools to control the browser slaves and uses zope.testrunner
> layers to ensure cleanup and also to share browser slaves between
> tests (massive speedup).  The final value is that developers get JS
> test failure feedback in bin/test.
> 

Test Runner:
    test'em was choice back then because it was solving some of the
issues we
    had with buster, but was still not perfect since it was hard to get
    coverege. research for this was done and at last sprint in oshosh we
    settled to give a try on Karma (known as Testacular before).

    just this week we switched from testem to carma

    why Karma?
     - its framework agnostic (use Mocha,Jasmine,QUnit,...)
     - it gives use coverage with a simple option which you need to turn
on and
       reports for text/html/lcov/cobertura
     - opens browsers and hooks them automatically so no magic code needed
       (remember when eric lost a day to get IE to open and run buster
tests?)
       you can also hook in other browsers manually (as buster does it)
     - can be hooked into selenium grid which means that it works with
       SauceLabs OOTB !!!!
     - integrates nice with other tools/services like Grunt, Travis,
       Jenkins, ...
     - Karma is also used by Angular-JS folks which means it has huge
       community behind it and awesome stuff pop out every month.
     - and most important ... it has all of this documented here
       http://karma-runner.github.io

Test Framework:
    Test runner (above) call test framework. I chose Mocha for following
    reasons:
     - you can run tests in standalone html page and it also comes with
console
       script
     - select your own style of testing: BDD/TDD (actually whole assertation
       library is pluggable and there are quite some interesting options to
       chose from, look next point about Test Assertation Library)
     - supported but many tests runners so its easy to get it running in CI
     - documentation is again great: http://visionmedia.github.io/mocha/

Test Assertation Library:
    A lot of testing frameworks dont give you the possibility to chose your
    own prefered style of assertations calls. Mocha above is different and
    gaves us choice to do this. I picked Chai assertation library, because:
     - its readable (you can chose between BDD/assert way of doing your
       assertations)
     - it has amazing documentation: http://chaijs.com

Other things we use in mockup and are part of our tests:
 - RequireJS: we use it extensivly in the code and we need test tools to
        support this
  http://requirejs.org/
 - SinonJS: Mocking library and a lot more
   http://sinonjs.org/


> > AFAIK, the current goal for Plone Core is to develop and test all JavaScript
> > in Plone Mockup (and then simply release packaged versions in core packages
> > and maybe test them once more with functional robot tests).
> >
> > So, currently there's no prepared story for testing add-on JS. However,
> > since mockup has public CI-integration on Travis-CI, it can be used as an
> > example.

we've also talked about this few days ago.

we'll use Yo scaffolding tool (from Yeoman, http://yeoman.io) to provide
scaffold which will setup addon project with same testing configuration
as we
have in mockup.

why we wont use any python scaffolding tools plone developers are familiar
with? mockup project currently only requires git and npm(node) to work
with it
and we dont was to add dependency (python) to that list. also developers we
target with mockup are not plone developers, but common web developers
who know
how to work with HTML/CSS/Javascript.


if you have any questions please first give a look at mockup repository
to get
the feel of how things are working and send questions to
plone-developers so we
can discuss them in open.


--
Rok Garbas - http://www.garbas.si




More information about the Plone-testing-team mailing list