[Setup] Re: Zope/Plone scalability

[Ricardo Newbery]

Adam,

I'm sure you're feeling frustrated but please be civil.


On May 9, 2008, at 5:40 PM, Getchell, Adam wrote:
>> Sure - badly behaved products suck.
>
> So, enterprise authentication using CAS isn't important to a CMS?


This is not what Martin said and I doubt it was even implied.



>> But this has nothing whatsoever to do with scalability or the
>> difference
>> between a ZEO server and a ZEO client.
>
> Well, leaving out the products, Plone/Zope still doesn't scale  
> (except possibly with ZEO Raid, which is in SVN so not looking  
> production-ready yet) to more than one backend natively, and its  
> performance is known to be poor, see for example [1], [2].


You do realize that your first citation is over 3 years old?  And the  
second citation appears to be mostly just a howto on setting up some  
monitoring tools if you've got a problem Zope instance -- it does not  
appear to make any claim that suggests that Zope's "performance is  
known to be poor".

I've never experienced a deployment that needed anything like ZEO Raid  
or RelStorage.  As has been pointed out before, it seems unlikely that  
your performance issues would be due to ZEO server bottlenecks.  Is  
there a specific reason that you feel this is your problem?

Note that I'm not claiming that Zope may not still have some kinks in  
the scalability department but it certainly seems to scale reasonably  
well enough to me and I've worked on some fairly demanding deployments.



>> 20+ Plone sites in one Zope instance is probably not a good idea,
>> unless
>> those 20+ sites are related in some way and share the same set of
>> products.
>
> They do.
>
>>> Should we be separate out more instances (thus, ZEO backends), or
>>> servers?
>>
>> Probably.
>>
>>> That's the aforementioned scalability issue.
>>
>> What is?
>
> The fact that you have to put Plone instances on different boxes. I  
> had no problems running 20+ websites on even a single IIS box, and  
> those could be easily load-balanced. If database driven with a SQL  
> backend, SQL could be clustered as well.


Nobody said anything about different "boxes" -- just different  
"instances".   In any case, you're talking about a different kind of  
"scalability".  Being able to scale up to multiple Plone instances in  
a single Zope is not really a performance issue as much as a  
management issue.  Let's not confuse the issues.  Perhaps in your  
case, the management issue is minimal.  Only you can determine that.



>>> But yes, we already carved out our main site and put it on a  
>>> separate
>>> box, just to see if we could make sense of the errors and isolate  
>>> the
>>> problem.
>>
>> Good.
>
> But, here's where I think we're seeing past each other -- I  
> *shouldn't* have to.
> One server -- one site doesn't scale well. That why I refer to it as  
> a "scalability" issue.


I suspect Martin was suggesting that this is "good" as a step toward  
isolating your problem.  If you're still getting these errors with a  
single Plone instance, it's not really a scalability issue (in either  
sense), is it?



>> I don't wish to be negative, but if you've spent "a couple of hundred
>> hours" and you haven't get got to the point where you've put a pdb in
>
> Not me, personally. If I'd have known the criterion for using Plone  
> was intimate knowledge of Python debugging, I wouldn't have asked my  
> web developers or system admins to use it.
>
>> that line and figured out what the problem is (or paid someone to  
>> do it
>> for you, if you don't know how), you need to look at how you're
>> investing your time and think about how you can get outside help in  
>> to
>> help you diagnose the problem before you sink any more cost into it.
>
> Right, so you're saying "Just fire up your Python debugger and go --  
> I'm not going to help you and if you can't help yourself or pay  
> someone, don't use Plone."
>
> Got it. You're right, I'll have to decide if it's worth my time.
>
> We just had an RFP for content management systems for our entire  
> campus. Plone was suggested, but not picked. I honestly would have  
> tried to change that decision before, but I see that it was a wise  
> one after all.


What is your point?  Clearly if your group has spent a couple hundred  
hours on this problem with no resolution, you need help.  This is not  
a meant as a slight.  A couple hundred hours is already costing you...  
either money or time or both.  Sometimes even the experts need help to  
figure something out.  Sometimes, debugging software is hard.  This is  
not a hurdle exclusive to Plone.  If irc or other community resources  
can't help you, you either need to pick up the skills or hire someone  
to do it.  There is no magic shortcut.

Ric