[Product-Developers] Re: Running all Plone tests?
optilude+lists at gmail.com
Sat Jan 10 17:27:38 UTC 2009
Chris Rossi wrote:
> I guess what I'm trying to glean, though, is whether it is a normal
> thing to want to run all of the tests and whether there is an
> expectation that they will all pass in a vanilla install.
Yes, there is. If a test is broken on a vanilla install, please consider
helping to fix the code or the test and committing back! As far as I
know, the release manager runs all the tests prior to making a release.
When I've run all the tests in the past, I've tended to enumerate over
all the packages in src/ in the coredev ploneout and run them
$ cd src ; packages=$(ls -1) ; cd ..
$ for x in $packages; do ./bin/instance test -s $x ; done
On trunk, I think you can just do:
$ ./bin/instance test -s plone
and run all the plone.* and plone.app.* tests, and probably with a few
more commands you can run most of our tests like that.
However, most people don't run all the tests all the time whilst they
are developing third party components, and if one of your third party
components breaks a Plone test when it's installed, it's not 100%
certain that this is because the component broke Plone - it may just
have altered some assumptions that the Plone tests made that are not key
to the test.
Conversely, your component could break Plone, but the test setup for
Plone's test may cause your package's changes to be ignored/not loaded
(this is kind of preferable - it means the package's tests are much more
like unit tests).
> If there is
> no expectation of such, amongst the community, then I'm not inclined to
> investigate further why individual tests are failing. I am perfectly
> happy to just run the tests for my packages and leave well enough
> alone. If I were doing coding that affected the core, though, I would
> be more concerned, since I'd be wondering how to establish a good
> baseline for regression testing.
All core tests should pass, of course.
> To ask in a different way, when Mr. Aspeli, in his tutorial on testing,
> says to "run all of the tests" what does he mean by "all"? All of the
> tests for your package? Or, really, all of the tests?
Heh, Mr. Aspeli means all the ones for the code you're working on.
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book
More information about the Product-Developers