[Product-Developers] Running all Plone tests?

[Chris Rossi]

I guess what I'm trying to glean, though, is whether it is a normal thing to
want to run all of the tests and whether there is an expectation that they
will all pass in a vanilla install.  If there is no expectation of such,
amongst the community, then I'm not inclined to investigate further why
individual tests are failing.  I am perfectly happy to just run the tests
for my packages and leave well enough alone.  If I were doing coding that
affected the core, though, I would be more concerned, since I'd be wondering
how to establish a good baseline for regression testing.

To  ask in a different way, when Mr. Aspeli, in his tutorial on testing,
says to "run all of the tests" what does he mean by "all"?  All of the tests
for your package?  Or, really, all of the tests?

Thanks!
Chris


On Fri, Jan 9, 2009 at 10:42 PM, Nathan Van Gheem <vangheem at gmail.com>wrote:

> I'll take a crack at this question; although, I wouldn't say that I really
> know anything as to why the tests wouldn't pass for you.
>
> To me, tests are run on a per-module basis.  The tests are most important
> for developers and documentation of the module.  Not sure why they wouldn't
> pass when being run all at the same time.
>
> Have you tried running one of the tests that was giving errors in
> isolation?
>
> On Fri, Jan 2, 2009 at 3:50 PM, Chris Rossi <chris at christophermrossi.com>wrote:
>
>> Hello all,
>>
>> I think this is probably mostly just a question about standard practice
>> and expected outcomes as regards testing in Plone.  I thought it would be
>> cute to try running all of the tests in Plone.  To do this I'm using the
>> most basic, undoctored buildout from ZopeSkel and then:
>>
>>  $ bin/instance test
>>
>> What I find is this results in a decent number of errors and doesn't
>> actually finish--the last output is a stack trace from one of the tests and
>> not a report of any kind.
>>
>> Which, makes me wonder, does anybody do this?  Is it advisable to even
>> think about doing this?  Is the standard practice to just isolate and test
>> the modules that are relevant to what you're doing?
>>
>> While it would be nice to have a nice, known to work, global baseline, I
>> understand that maybe some of these tests assume external dependencies in
>> your server environment are configured in a particular way.  What is the
>> standard practice, here, with regard to running these tests and what is the
>> expected outcome?
>>
>> Running the above generated about 4MB of console output that I'm happy to
>> forward to anyone who cares take a gander.
>>
>> Thanks,
>> Chris Rossi
>>
>>
>> _______________________________________________
>> Product-Developers mailing list
>> Product-Developers at lists.plone.org
>> http://lists.plone.org/mailman/listinfo/product-developers
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.plone.org/pipermail/plone-product-developers/attachments/20090110/b03df744/attachment-0001.html>