You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The attached test script and output file appear to show logging working
correctly. The script writes a log message including the Cyrillic text
доброе утро (Good morning) to a CP1251-encoded file, test.log. Opening
this file in a Unicode-aware editor (I used BabelPad, http://www.babelstone.co.uk/Software/BabelPad.html) appears to show the
Cyrillic characters correctly: see the screenshot at
Traceback (most recent call last):
File "e:\bin\python\0\lib\logging\__init__.py", line 765, in emitself.stream.write(fs % msg.encode("UTF-8"))
File "e:\bin\python\0\lib\codecs.py", line 686, in writereturnself.writer.write(data)
File "e:\bin\python\0\lib\codecs.py", line 351, in write
data, consumed =self.encode(object, self.errors)
File "e:\bin\python\0\lib\encodings\cp1251.py", line 12, in encodereturn codecs.charmap_encode(input,errors,encoding_table)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 0:
ordinal not in range(128)
Sorry, misread the original issue and tested with Python 2.5 rather than
2.6. This is a regression; fix and additional test case checked into
trunk and release26-maint.
Can you retry with setting the "encoding" attribute of the file to
"cp1251"? That should work and that should be the appropriate method to
avoid the problem.
test_logging.py in the Python distribution has a test which exercises
Unicode functionality using cp1251, does that test work in your environment?
shamilbi commented on Apr 21, 2009
shamilbimannequin
MannequinAuthor
More actions
Can you retry with setting the "encoding" attribute of the file to
"cp1251"? That should work and that should be the appropriate method to
avoid the problem.
test_logging.py in the Python distribution has a test which exercises
Unicode functionality using cp1251, does that test work in your
environment?
logging to file is OK (since 2.6.2), logging to console __was__ OK (2.6.1),
not now (2.6.2)
Trunk and release26-maint were recently changed (r71657, r71658) to use
the following logic, which differs from the code snippet you posted.
if (isinstance(msg, unicode) and
getattr(stream, 'encoding', None)):
stream.write(fs.decode(stream.encoding) % msg)
else:
stream.write(fs % msg)
If the stream is stderr and you are passing a unicode msg, the else
branch will not be taken; as long as the stream has an encoding
attribute, it should output correctly.
The change was made when another, similar issue was posted by another
user (issue bpo-5768).
Can you confirm what happens with the current code as it is in
release26-maint? спасибо!
shamilbi commented on Apr 22, 2009
shamilbimannequin
MannequinAuthor
More actions
Trunk and release26-maint were recently changed (r71657, r71658) to use
the following logic, which differs from the code snippet you posted.
if (isinstance(msg, unicode) and
getattr(stream, 'encoding', None)):
stream.write(fs.decode(stream.encoding) % msg)
else:
stream.write(fs % msg)
If the stream is stderr and you are passing a unicode msg, the else
branch will not be taken; as long as the stream has an encoding
attribute, it should output correctly.
The change was made when another, similar issue was posted by another
user (issue bpo-5768).
Can you confirm what happens with the current code as it is in
release26-maint? спасибо!
it still doesn't work for console (but OK for files).
the following works in both cases:
if (isinstance(msg, unicode) and
getattr(stream, 'encoding', None) and
(stream == sys.stdout or stream == sys.stderr)):
stream.write(fs % msg.encode(stream.encoding))
else:
stream.write(fs % msg)
i think it's all about the difference betwin print(msg) and
sys.stdout.write('%s\n' % msg)
i think it's all about the difference betwin print(msg) and
sys.stdout.write('%s\n' % msg)
Yes. Actually, it's about what streams will accept. For example, a
stream opened with codecs.open(encoding='cp1251') will accept a Unicode
string and encode it before output. Likewise, a StringIO wrapped inside
a writer obtained from the codecs module.
However, stdout and stderr are of type file and seem to behave
differently - they behave as expected if I encode the value written to
them, but fail if I write a Unicode string to them.
I don't think special-casing for sys.stdout and sys.stderr in the
logging code is the correct approach. Better to put a conditional
encoding step when it's needed - now I just need to figure out all the
cases when it's needed ;-)
Activity
shamilbi commentedon Feb 6, 2009
if i configure logging into a file with encoding = 'cp1251' and do
logger.debug(u'...') then i get crash with UnicodeError
i suggest reimplementing method FileHandler.emit():
...
if isinstance(msg, unicode):
stream.write(f % msg) # it works!
...
vsajip commentedon Feb 6, 2009
The attached test script and output file appear to show logging working
correctly. The script writes a log message including the Cyrillic text
доброе утро (Good morning) to a CP1251-encoded file, test.log. Opening
this file in a Unicode-aware editor (I used BabelPad,
http://www.babelstone.co.uk/Software/BabelPad.html) appears to show the
Cyrillic characters correctly: see the screenshot at
http://img5.imageshack.us/img5/799/cp1251zw2.png
Marking as pending. Can you give more specifics about your problem?
shamilbi commentedon Feb 8, 2009
test_log.py:
-----------
#! -- coding: windows-1251 --
exception:
---------
vsajip commentedon Feb 8, 2009
Sorry, misread the original issue and tested with Python 2.5 rather than
2.6. This is a regression; fix and additional test case checked into
trunk and release26-maint.
shamilbi commentedon Apr 20, 2009
(python 2.6.2, WinXP)
logging to console stopped working. Here is a workaround:
logging/__init__.py: class StreamHandler(Handler): ... def emit(self, record): ... if (isinstance(msg, unicode) or getattr(stream, 'encoding', None) is None): - stream.write(fs % msg) + if stream == sys.stdout: + print(msg) + else: + stream.write(fs % msg)vsajip commentedon Apr 21, 2009
Can you retry with setting the "encoding" attribute of the file to
"cp1251"? That should work and that should be the appropriate method to
avoid the problem.
test_logging.py in the Python distribution has a test which exercises
Unicode functionality using cp1251, does that test work in your environment?
shamilbi commentedon Apr 21, 2009
logging to file is OK (since 2.6.2), logging to console __was__ OK (2.6.1),
not now (2.6.2)
shamil
vsajip commentedon Apr 21, 2009
Trunk and release26-maint were recently changed (r71657, r71658) to use
the following logic, which differs from the code snippet you posted.
If the stream is stderr and you are passing a unicode msg, the else
branch will not be taken; as long as the stream has an encoding
attribute, it should output correctly.
The change was made when another, similar issue was posted by another
user (issue bpo-5768).
Can you confirm what happens with the current code as it is in
release26-maint? спасибо!
shamilbi commentedon Apr 22, 2009
it still doesn't work for console (but OK for files).
the following works in both cases:
if (isinstance(msg, unicode) and
getattr(stream, 'encoding', None) and
(stream == sys.stdout or stream == sys.stderr)):
stream.write(fs % msg.encode(stream.encoding))
else:
stream.write(fs % msg)
i think it's all about the difference betwin print(msg) and
sys.stdout.write('%s\n' % msg)
shamil
vsajip commentedon Apr 22, 2009
Yes. Actually, it's about what streams will accept. For example, a
stream opened with codecs.open(encoding='cp1251') will accept a Unicode
string and encode it before output. Likewise, a StringIO wrapped inside
a writer obtained from the codecs module.
However, stdout and stderr are of type file and seem to behave
differently - they behave as expected if I encode the value written to
them, but fail if I write a Unicode string to them.
I don't think special-casing for sys.stdout and sys.stderr in the
logging code is the correct approach. Better to put a conditional
encoding step when it's needed - now I just need to figure out all the
cases when it's needed ;-)
vsajip commentedon Apr 22, 2009
Fix checked into trunk and release26-maint.