[Product-Developers] Plone string encoding problem

[Raphael Ritz]

On 12/22/11 11:44 AM, mcelotti wrote:
> I have a Document (type_name=Document) containing this text:
> "accents è à ì ò ù"
>
> When I open the doc everything is fine and all the accents are correct.
>
> The problem comes with my custom script:
>
> items = context.portal_catalog(portal_type='Document',
> UID='a64ce6ceb295bcaa48ebff874379d907')
> array=[]
> for item in items:
>    dict={}
>    obj = item.getObject()
>    dict['text']= obj.getText()
>    array.append(dict)
> return array
>
> The output is:
> "accents \xc3\xa8 \xc3\xa0 \xc3\xac \xc3\xb2 \xc3\xb9"
>
> I need to write original content to database (mysql utf-8) but I'm stuck
> with this problem ...

What problem?
What you see above is indeed a utf-8 encoded version
of the string you started out with:

Python 2.6.6 |EPD 6.3-2 (32-bit)| (r266:84292, Sep 23 2010, 11:52:53)
[GCC 4.0.1 (Apple Inc. build 5488)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> s = u"accents è à ì ò ù"
 >>> s
u'accents \xe8 \xe0 \xec \xf2 \xf9'
 >>> s.encode('utf-8')
'accents \xc3\xa8 \xc3\xa0 \xc3\xac \xc3\xb2 \xc3\xb9'

Maybe you are confused by 'print' masking this?

 >>> print s
accents è à ì ò ù
 >>> print s.encode('utf-8')
accents è à ì ò ù
 >>>


Raphael

>
> Also, if I modify my script this way (print output):
>
> items = context.portal_catalog(portal_type='Document',
> UID='a64ce6ceb295bcaa48ebff874379d907')
> array=[]
> for item in items:
>    dict={}
>    obj = item.getObject()
>    dict['text']= obj.getText()
>    print dict['text']
>    array.append(dict)
> return printed
>
> Everything is fine and I can see the correct text with accents.
>
> Thank you,
> Marco
>
> --
> View this message in context: http://plone.293351.n2.nabble.com/Plone-string-encoding-problem-tp7118197p7118197.html
> Sent from the Product Developers mailing list archive at Nabble.com.
> _______________________________________________
> Product-Developers mailing list
> Product-Developers at lists.plone.org
> https://lists.plone.org/mailman/listinfo/plone-product-developers