[Product-Developers] z3c.form UnicodeDecodeError

Daniel Widerin daniel.widerin at kombinat.at
Tue Mar 31 14:12:35 UTC 2009


hi z3c.form gurus,

i've successfully implemented some z3c.forms for custom AT types into 
plone. They work fine, but i noticed that entering a non-ascii character 
like 'Ö' for Österreich the default BaseDataConverter raises a 
UnicodeDecodeError in z3c.form.converter.

   ...
   Module z3c.form.form, line 196, in __call__
   Module z3c.form.form, line 191, in update
   Module z3c.form.form, line 141, in update
   Module z3c.form.form, line 133, in updateWidgets
   Module z3c.form.field, line 259, in update
   Module z3c.form.browser.text, line 35, in update
   Module z3c.form.browser.widget, line 61, in update
   Module z3c.form.widget, line 119, in update
   Module z3c.form.converter, line 49, in toWidgetValue
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: 
ordinal not in range(128)

It tries to return unicode(value).
When i changed this statement into return unicode(value.decode('utf-8')) 
it worked - could there be an issue with the charset/encoding when 
ATFieldProperty is used? maybe there's a encoding difference between 
accessor and atfieldproperty?

After changing this, i'm able to view the form, but not able to save it, 
because in z3c.form.form in applyChanges again a UnicodeDecodeError is 
raised. I've played around with the debugger:

(Pdb) l
  46
  47
  48             import pdb; pdb.set_trace()
  49
  50             # Only update the data, if it is different
  51  ->         if dm.get() != data[name]:
  52                 dm.set(data[name])
  53                 # Record the change using information required later
  54                 changes.setdefault(dm.field.interface, []).append(name)
  55         return changes
  56
(Pdb)
-> if dm.get() != data[name]:
(Pdb) dm.get()
'B\xc3\xbcndtlittenstrasse 3a'
(Pdb) data[name]
u'B\xfcndtlittenstrasse 3a'
(Pdb) isinstance(dm.get(), unicode)
False
(Pdb) isinstance(data[name], unicode)
True
(Pdb) unicode(dm.get())
*** UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 
1: ordinal not in range(128)
(Pdb) unicode(dm.get().decode('utf-8'))
u'B\xfcndtlittenstrasse 3a'
(Pdb) data[name] == unicode(dm.get().decode('utf-8'))
True
(Pdb)

i had a look into z3c.form 2.0 but could not see any fix in there. So - 
i think i'm doing something wrong here, could anybody point me into the 
right direction?

thanks





More information about the Product-Developers mailing list