Posted By: Gregg Lind
I’m having some brain failure in understanding reading and writing text to a file (Python 2.4).
# The string, which has an a-acute in it. ss = u'Capitxe1n' ss8 = ss.encode('utf8') repr(ss), repr(ss8)
print ss, ss8 print >> open('f1','w'), ss8 >>> file('f1').read() 'Capitxc3xa1nn'
So I type in
Capitxc3xa1n into my favorite editor, in file f2.
>>> open('f1').read() 'Capitxc3xa1nn' >>> open('f2').read() 'Capit\xc3\xa1nn' >>> open('f1').read().decode('utf8') u'Capitxe1nn' >>> open('f2').read().decode('utf8') u'Capit\xc3\xa1nn'
What am I not understanding here? Clearly there is some vital bit of magic (or good sense) that I’m missing. What does one type into text files to get proper conversions?
What I’m truly failing to grok here, is what the point of the UTF-8 representation is, if you can’t actually get Python to recognize it, when it comes from outside. Maybe I should just JSON dump the string, and use that instead, since that has an asciiable representation! More to the point, is there an ASCII representation of this Unicode object that Python will recognize and decode, when coming in from a file? If so, how do I get it?
>>> print simplejson.dumps(ss) '"Capitu00e1n"' >>> print >> file('f3','w'), simplejson.dumps(ss) >>> simplejson.load(open('f3')) u'Capitxe1n'
In the notation
the “xe1” represents just one byte. “x” tells you that “e1” is in hexadecimal.
When you write
into your file you have “xc3” in it. Those are 4 bytes and in your code you read them all. You can see this when you display them:
>>> open('f2').read() 'Capit\xc3\xa1nn'
You can see that the backslash is escaped by a backslash. So you have four bytes in your string: “”