[Setup] migration 2.0.5->2.1.1 troubles

[Nick Davis]

   We have something in our zodb, that causes migration to break, but we 
don't know where.

   Platform is Zope 2.7.8 / Python 2.3.5 / Debian Sarge

   If I start with empty Data.fs, create a new Plone site, and try to 
migrate it, it works. If I add a few things to an empty Plone site like 
a user, a image or two, a couple of ZWiki pages, and migrate that, that 
works.
   This implies our environment is OK.

   If I try to migrate our whole site, with or without 3rd-party 
products installed, it repeatedly says Recataloging object failed, 
because things such as enable_livesearch, typesUseViewActionInListings 
etc that apparently should have been added by the migration code, have 
not been added. It appears an earlier stage of the migration stumbled on 
some data but instead of noisily error-ing out as it should, simply 
skipped several steps which had a knock-on effect later.

   So, starting with a copy of our live Data.fs, I deleted all new 
directories that we'd created under the plone root, uninstalled all 
3rd-party products, and rebuilt portal_catalog, reference_catalog (don't 
know how important that is).
   I then tried to migrate this. One would think after wiping so much 
out, this would resemble an empty Plone site and migrate OK, but it does 
not. Again some step(s) are missed, and when trying to migrate simple 
objects such as Member's index.html s that have never been edited, it 
can't do it because other migration code appears not to have run.

   I am trying to debug the migration code, which isn't easy because its 
complicated.

   Also I am considering somehow dumping out some of our data then 
adding it back in to a fresh site. If anyone can point me to a HOWTO 
that would be good.

   Our Data.fs shouldn't be in too bad a shape. We are biased towards 
doing stuff on the filesystem not ZMI. We did rename our Plone root at 
one point. I can't think of any other risky things we've done though.

   This has taken a lot of time and has been so frustrating I have 
wondered about ditching Plone and writing something of our own on top of 
Zope/CMF even if that is a lot of coding work.

   What else should I look for in the zodb that might be breaking things?

   How thoroughly tested is the migration code?

   Can anyone shed some pearls of wisdom or make suggestions of where to 
go from here?

   Thanks

   Nick