Migration profiles for add-on products?

Martin Aspeli optilude at gmx.net
Fri May 11 23:59:13 UTC 2007


*ARGH* .... mis-sent this (again, I'm doing that a lot lately, my 
fingers must be getting fatter!).

I was half-way through refactoring my fake code...

>> class Migration(object):
>>      implements(IMigration)
>>      def __init__(self, id, to_version, profile):
>>          self.id = self
>>          self.to_version = version
>>          self.profile = profile
>>
> 
> Let's compare this to my untested code in
> https://svn.plone.org/svn/collective/collective.generations/trunk
> 
>   from Products.collective.generations import SchemaManager
> 
>   sm = SchemaManager(
>       minimum_generation = 0,
>       generation = 5,
>       package_name = 'Products.eXtremeManagement'
>       )

I think the name SchemaManager is a bit misleading, though. Is it
managing an Archetypes schema? :)

We are still relying on naming conventions. Also, the minimum_generation
thing is for zope.generation's auto-migration thing, which I don't think
can (or should) have an analogue in Plone.

The other problem here is that users don't really think of
"generations", they think of versions, which are sequential things.

We could solve this by being a bit more explicit:

class IVersionManager(Interface):
      """Keep track of the current version for a product
      """

      current_version = schema.TextLine("The current version")

      product_name = schema.TextLine("Describe the product")

      migration_steps = schema.Tuple("An ordered list of migration steps")

class IMigrationStep(Interface):
      """Describes a migration
      """

      to_version = schema.ASCIILine("Migrates to version - displayed to
the user")

      def __call__(portal):
          """Execute this migration step
          """

class VersionManager(object):
      implements(IVersionManager)

      def __init__(self, current, name, steps=[]):
          self.current_version = current
          self.product_name = name
          self.steps = steps

class GSMigrationStep(object):
       """A migration step using a GS profile
       """

       implements(IMigrationStep)

       def __init__(self, version, profile):
           self.to_version = version
           self.profile = profile

       def __call___(self, portal):
           portal_setup = getToolByName(portal, 'portal_setup')
           portal_setup.importProfile(self.profile)

... now, in e.g. Poi, I'd do:

version = VersionManager(
	name=u"Poi",
          steps=[
		GSMigrationStep("1.1", "profile-Products.Poi:1.1"),
		GSMigrationStep("1.2", "profile-Products.Poi:1.2"),
		GSMigrationStep("1.3", "profile-Products.Poi:1.2"),
	]
)

provideUtility(version, name=u"Products.Poi") # do this with ZCML

So, we have one global utility, named after the product by convention.
We specify the name of the product to display in the migration UI.

The migration steps are listed explicitly, in order. The "latest"
version is the last one in the list (remember, it specifies "versions
being migrated to"). That makes sense - it's the last thing you can
migrate to.

We don't depend on GS profiles only, but provide a default 
implementation that uses these, since it's probably most sensible.

In this, we reference a profile by name (we could simplify by putting 
the product name in the VersionManager, and/or defaulting the profile 
name to profile=${product_name}:${version}, but I think it makes sense 
to be able to specify one explicitly.

When we migrate current version to 1.3, say, the migration manager goes 
through the list from the current/pre-migration version, and calls each 
step up until and including the last one.

Adding a new migration step means adding another element to that list,
which should be pretty simple.

To me, this is easier to understand and manager than sequential ints
that have no tie to the actual version number I'm releasing, and will
make more sense to users.

Of course, you may then end up with "pseudo-versions" if you have a need
for migrations in-betwen releases, but I think this is a minor need, and
that people should be making more releases anyway. :) But we do need to
make sure it's possible to "force migrate" from a version.

The missing pieces now are:

   - a local utility which keeps track of the current/last migrated
version of a package

   - global utility or adapter that implements the service of running 
migrations

   - some standard way of refreshing this if a product is re-installed
from scratch

   - UI for this, including the ability to force-migrate from a given 
version

In general, I don't like encoding meaning into names of things. It's too
easy to get it wrong. Also, I find strings that contain version numbers
really hard to read and easy to get wrong: "migrate-1.1a2-2.0rc1" - yipes!

>> then, you register a profile, like Products.Poi:migrate-1.0-2.0 and
>> Products.Poi:migration-2.0-2.1
> 
> That would be something like this:
> 
>   <genericsetup:registerProfile
>       name="3"
>       title="Migration profile for eXtremeManagement generation 2 to 3."
>       description="This profile contains configuration changes that
>                    are applied during the migration from
>                    eXtremeManagement generation 2 to 3, which is part
>                    of the change from 1.1 to 1.5."
>       directory="migrations/v1.5"
>       provides="Products.GenericSetup.interfaces.MIGRATION"
>       />
> 
> Note that Products.GenericSetup.interfaces.MIGRATION does not
> currently exist.  It is just an idea.  But that is not really an issue
> here.

It can be collection.migration.interfaces.IMigrationProfiles. No need
for it to be in CMF Core. :)

> No utilities need to be registered with the collective.generations
> approach.  That one SchemaManager should be enough.  Maybe that can be
> done in zcml though if wanted.

I don't see any benefit of doing this with ZCML at this point, just more
work for you. :)

>> It ought to be an explicit action in Install.py when reinstall=True, but
>> that sounds sensible, yes.
> 
> Extensions/install.py could have something like this:
> 
>   from Products.collective.generations import UpgradeManager
>   from Products.collective.generations import SchemaManager
> 
>   sm = SchemaManager(
>       minimum_generation = 0,
>       generation = 5,
>       package_name = 'Products.eXtremeManagement'
>       )
> 
> 
>   def install(context):
>       um = UpgradeManager(context) # or getUtility/getToolByName
>       um.install(sm)

It may make more sense if this was either an adapter or a utility,
rather than a helper class. I think a utility makes more sense, even if
we have to pass in the context:

migrations = getUtility(IMigrationManager)
migrations.migrate_to_current_version(portal, "Products.Poi")

> And during a reinstall it is probably enough to evolve from the
> current iteration to the maximum defined generation, using this
> method:
> 
>       um.evolve_maximum(sm)
>       
> I do not know if this works, but I *am* beginning to like the
> approach. :-)

I'm beginning to like it less. :) The more I think about it, the more
wrong it seems to tie this to an arbitrary, incremented integer. Users
and developers think in terms of releases, which have unique and
sequential versions. There's lots of scope for things to go subtly
pear-shaped during migration, and being explicit and keeping things
simple seems a good general principle.

Also, I'm not sure I see the need for the min/max generation stuff
(which is easier to do with integer generations, for sure). From what i
remember, zope.generation does it so that you can do auto-migration on
Zope startup for "Zope won't even run properly if you don't do this"
kind of migrations. That's fine for fundamental Zope things, but I don't
think auto-migration is *ever* a good idea in the context of Plone, let
alone technically difficult (when do you trigger it?).

Besides that, the only migration that ever makes sense to invoke
explicitly is from "current version" to "latest version", with a
fallback option of explicitly choosing the "current version" if the
system's lost track. This is how Plone's migration has worked for years,
and I think this is how people expect it to be.

Thoughts?

Martin





More information about the Product-Developers mailing list