[Framework-Team] Re: PLIP load test reports available

[Maurits van Rees]

On Sun, Sep 06, 2009 at 05:52:52PM +0800, Martin Aspeli wrote:
> This is really good stuff!
>
> I must admit, I don't understand the graphs or the tables-of-graphs at
> all, though. Is there a quick explanation somewhere about how to read
> them?

The reference bench results are for Plone4 without plips.
The challenger bench results are for one of the plips.

If the graph is mostly green, the plip has better performance.
If the graph is mostly red, the plip has worse performance.

For each plip three tests have been done, showing performance for:
- content creation
- read only
- heavy writes
I don't know what is being tested exactly.

On the front page of the tests at
http://weblion.psu.edu/static/loadtesting/plone4.0/plips.html
you have per plip the following items:
- label/name of the plip
- for each of the three tests:
  - main graph, with link to detailed report of the plip
  - main difference graph between plip and plain plone, with link to
    detailed difference report

Take the diffence page for plip 9310 showing heavy writes:
http://bit.ly/vN9kJ

The first graph has the requests per second.
The blue line shows the results for B1 (bench 1, plain plain 4.0)
The purple line shows the results for B2 (bench 2, plip 9310)

The first part shows the difference between those lines as red,
meaning that the plip is slightly slower; this part of the graph is
for 1-3 concurrent users.

To the right we have a green difference, meaning that the plip is
faster, even up to 80% for 10 concurrent users.

If results look too absurd, probably something went wrong in the
tests.  For example plip 8814, replacing secure mail host, looks
totally red.  Apparently with that plip we serve a whopping zero pages
per second. :-)

I hope that clears things up a bit.

-- 
Maurits van Rees | http://maurits.vanrees.org/
            Work | http://zestsoftware.nl/
"Yes, we study economics,
got to make the money make some kind of sense."  - Geoff Mann