[Framework-Team] Re: ploneenv - Or how using workingenv for a common Zope2 project might look like ;-)

Martin Aspeli optilude at gmx.net
Sun Feb 4 01:03:20 UTC 2007


 > I'm obviously for ploneenv/workingenv.

I'm obviously a bit biased towards the buildout based approach since I 
worked on it, but I worked on it because I was never very happy with the 
way workingenv-in-instances worked. ploneenv makes that better and 
slicker, actually, and I quite like it. I'm not really trying to argue 
for or against anything, since it may largely come down to individual 
preference. My post was mostly to try and explain what I feel are some 
of the differences and similarities to allow others to work out what's 
appropriate for them. I think your comments work towards this as well.

Btw, can we move at least the Plone egg to svn.plone.org? I suggest 
svn.plone.org/svn/dist/Plone or something similar, since as you pointed 
out, /Plone may be a little confusing. If this really is to become a 
"meta-egg" with just dependencies, then I think something like /dist 
make sense.

 > I wouldn't have developed ploneenv otherwise.

Some of this also happened in parallel, I think. :)

> I believe that "the whole of zc.buildout and ploneout" is *a lot* bigger
> compared to workingenv and ploneenv in lines of code.  Unless you mean to
> compare the buildout *recipes only* with both workingenv and ploneenv, which
> seems unfair.  For the reasons of compactness and transparency, I expect
> ploneenv to be more maintainable.

I really can't be bothered to count lines of code, but compared to most 
of the other stuff we deal with, zc.buildout is not very big and it's 
well-maintained.

> Anyway, the main reason I think that workingenv is simpler is because it's
> transparent to the user.  By user I mean someone who has used distutils
> before (i.e. "python setup.py install") and someone who wants who has done
> Zope instances before and wants to reapply his experience.
> 
> ploneenv is easier for developers because it lets them use Zope instances
> the way they're used to use them.  That is, "bin/zopectl" works

Well, so it does in ploneout; it's in parts/instance/bin/zopectl, 
though. And there's a top-level script bin/instance which works the 
same. I don't think that's particularly an obstacle.

>, "Products/"
> works as expected etc.

Again, so it does in ploneout, but we prefer a top-level products/ 
folder (which works identically to parts/instance/Products). This may be 
an issue, I agree, when documentation says $INSTANCE_HOME/Products. It's 
done this way to offer better separation between "code you write" and 
"stuff that's needed to get your code to run" that you don't necessarily 
worry about.

>  And there is no way that workingenv could decide to
> delete any directory, which you might consider a pitfall with ploneout.

I don't consider it a pitfall, particularly, certainly not of ploneout. 
It was a problem with a first, naive implementation of one of the 
recipes. But the buildout model does mean that you re-run the buildout 
from time to time (when you change the configuration), which means that 
you could conceivably make changes that get stomped upon. The rule of 
thumb is "if it's in parts/ let buildout manage it, otherwise you're safe".

I suppose you could run buildout only once and then make changes 
manually thereafter, but that really removes the whole point of buildout. :)

> My impression was that ploneout would at some point grow a deployment
> configuration.

Not quite sure what you mean by this. We could use its recipes to make a 
buildout that could act as an installer for zope and plone, potentially, 
if there was interest. This may make it easier to cut releases, for 
example.

This is probably a place where the buildout.cfg approach makes sense - 
we would be able to edit this file so that at any point in time it 
pointed at the right eggs to construct a Plone release. We could then 
let buildout run additional recipes that would e.g. run the tests and 
then create eggs, upload them to the cheese shop, and/or roll tarballs. 
We would code that in python and package it as buildout recipe eggs.

> BTW, the Plone egg *does* exist[1].  And I'd like to move it to
> svn.plone.org :-)

Yeah, true, I meant in the "on the cheeseshop" sense. But that's great, 
and I really think we should move it to svn.plone.org.

> I've been working with buildout before[2], but I still find it hard to use
> (not as someone who only executes bin/buildout obviously, but as someone
> trying to make a testrunner script, for example).

I think the testrunner script is a particularly nasty example given how 
badly the standard testrunner deals with eggs. However, I think there is 
a sense that the zc.buildout model fits some developers better than 
others. I found it easy to understand and work with. I had expected it 
to be a lot worse. Whether I'm typical, I don't know. :) There is 
something a documentation issue here as well.

> Zope 2 turns out to be quite good with eggs.  I consider lacking egg support
> of the testrunner a bug.  It's a shame you have to fiddle with paths just to
> get your tests running.  Pretty please: someone fix the testrunner.

Indeed, that's mostly what I was talking about. That, and the fact that 
by default it doesn't consider the global PYTHONPATH.

>> The main difference, as I see it, is that zc.buildout (and thus by
>> extension ploneout) takes a step back from the notion of a Zope
>> instance. In the above, the Zope instance is one part. Well, the only
>> part here, but the Zope 2 installation itself could be another part, as
>> could 'zopepy', a python interpreter that is pre-configured to know all
>> about Zope's PYTHONPATH so that you can run scripts and interactively
>> execute code as if you were Zope (almost). If your environment has other
>> requirements, e.g. let's say you were using WSGI to serve up parts of
>> your application from another web server and you needed to build and
>> configure that, you'd need an appropriate recipe and then wire it into
>> your buildout.cfg.
> 
> I'd certainly look at the solutions that WSGI- land has grown before I would
> decide to use buildout for this.

Such as? Would you Pylons into $INSTANCE_HOME/lib/python for example, 
and paster into $INSTANCE_HOME/bin to use for 'paster serve'? I find 
this a bit odd, personally, since $INSTANCE_HOME is very much Zope's 
domain. Anyway, it was just an example. What if you need to build some C 
library or install binaries for MS Office transforms?

> Note that the z2c.recipe.zope2instance[3] recipe alone, which you consider
> "plain python" is bigger than the whole of ploneenv[4].

Fair enough, but it also does a lot more. Most of the complexity deals 
with making it easy to configure Zope. I don't think anything in there 
is particularly complex, hard to understand, or hard to maintain/debug. 
The only alien things are probably the way to inspect buildout options 
(the 'options' dict-like object) and the zc.recipe.egg thing (which just 
inspects buildout.cfg looking for eggs and lets you find them + 
dependencies).

> Support for Windows should be fairly trivial.  I would appreciate it if
> someone (Hanno?) gave it a try.  Basically, all we need is the correct way
> to patch bin/zopectl so that it runs the bin/activate.bat script before
> startup.  We shouldn't seriously consider "runs on Windows" as an argument
> for ploneout, just because ploneenv hasn't seen a Windows developer yet. :-)

No, agree. Mostly just a prod for someone to fix that. :)

> Because ploneenv does not worry about installing Zope, the whole platform
> independence story becomes simpler.  You can just use your Debian
> installation of Zope[6] for example, and benefit from the automatic updates
> that the system gives you.  Or you could use the existing Zope installer for
> the various platforms.

That's basically a choice you make; either use the 
z2c.recipe.zope2install recipe to get zope 2, or refer to an existing 
zope 2 installation if you wish. In the example above, we don't install 
Zope 2. On the other hand, I never let package managers install zope for 
me since I normally need lots of different versions around.

> Scripts are installed in $WORKING_ENV/bin

I thought as much. And $WORKING_ENV == $INSTANCE_HOME, right?

>>  - Multiple developers can share a single buildout.cfg (e.g. in svn) and
>> have a fully repeatable environment. This is essentially what ploneout
>> tries to do for Plone developers.
> 
> This implies that every Plone developer needs a full checkout of any Plone
> dependency, whereas with ploneenv / the Plone egg they can 1) use existing
> checkouts (which I like to do even across projects) and 2) use just the
> number of checkouts as you need.

That's no different. In the example above, each developer on a team 
would have a local copy of the Plone egg and its dependencies. Only eggs 
in development mode (which all the Plone eggs in ploneout are, because 
it's aimed a Plone core developers, but this would not be the case if 
you used a buildout for your own project as in the example above) would 
be bundled with the buildout and put in the src/ directory. Or not, if 
that wasn't appropriate.

The point here is that the buildout.cfg is *one* point of reference for 
the build configuration of the whole project (which may include things 
other than "install these eggs"), and it can be replicated/used across 
multiple environments. That to me is the main advantage of a buildout 
based approach. There are ways of making workingenv bootstrap in a set 
of eggs (--requirements) but from that point on, the configuration can 
change independently. That may or may not be a problem (probably less of 
a problem with good egg dependency management between the things being 
developed and their dependencies).

> Note that you can choose to not have your system-wide packages available in
> a ploneenv by giving it the --no-site-packages option.  The default is that
> you *have* access to those packages.

Cool. I like that default better than the other way around.

> Well, my experience is that with complex scripted installation, you can only
> *hope* that they're repeatable across platforms and OSes.  For something as
> widely in use as I imagine ploneout could be, it's less of a problem maybe
> to ensure cross platform compatibility, but for your own buildout?  My point
> is that using zc.buildout doesn't buy you platform compatibility.

There's cross-platform and there's cross-environment. Of course we had 
to do extra work to make ploneout work well on Windows. However, my 
ploneout will be the same as yours. So, for example, I find it pretty 
attractive to tell all the sprinters who don't already have Plone 3 (or 
their own ploneenv... :-)) to run the three ploneout commands (svn, 
bootstrap, buildout) to get an environment that's "known good". Imagine 
if you have a development team all with Macs or Linux boxes, for example.

Ultimately, I think we are arguing preferences and taste. The main lever 
for all of this is eggs (which we all love, of course :-)) and both 
approaches basically depend on eggs and setuptools as much as possible.

For things where non-Zope/Python things are part of the build 
environment, buildout clearly gives you additional functionality, which 
you may or may not need. At the end of the day, I don't think the 
complexity/maintainability argument should be cause for concern, and so 
it may come down to how much you like traditional zope instances and 
easy_install versus more explicit single-file configuration.

Martin





More information about the Framework-Team mailing list