[Product-Developers] The most efficient way to store 270 AT fields?

[Andreas Jung]

On 05.01.2009 13:18 Uhr, Mikko Ohtamaa wrote:
> Hi,
>
> We are facing a problem where we need to store 270 fields per item. The
> fields are laboratory measurements of a patient - 40 measurement values for
> 7 timepoint. The fields need to be accessed per timepoint, per measurement
> and all fields for one patient once. There will be over 10000 patients,
> distributed under different hospital items (tree-like, for permission
> reasons). Data is not accessed for two patients at once, so we don't need to
> scale the catalog.
>
> So I am curious about how we make Plone scale well for this scenario.
>
> - The overhead of a field in AT schema? Should we use normal storage backend
> (Python object value) or can we compress or field values into list/dict to
> make it faster using a custom storage backend.
>
> - The wake up overhead of AT object? Should we distribute our fields to
> several ZODB objects e.g. per timepoint, or just stick all values to one
> ZODB objects. All fields per patient are needed on some views once.
>
> - One big Zope objects vs. few smaller Zope objects?

Don't try to implement using one schema with 270 fields - that would be 
insane and broken-by-design. Do you really need schema support for 
rendering input forms? Keep in mind how a user would/should use a form 
with 270 fields. I would implement a "person" objekt as a folderish. 
Implement a Timepoint object (with the 40 fields) and stick them under 
with the related "persom" folder/object. If you are smart, drop 
Archetypes and implement something more lightweight on top of Z3 schemas 
and formlib.

-aj

-------------- next part --------------
A non-text attachment was scrubbed...
Name: lists.vcf
Type: text/x-vcard
Size: 316 bytes
Desc: not available
URL: <http://lists.plone.org/pipermail/plone-product-developers/attachments/20090105/db3e45f6/attachment.vcf>