question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Unicode decode error while writing log output

See original GitHub issue

Suppose I install virtualenv 1.10.1 with pip 1.4.1 on a Debian system with Python 2.7.4, but without the python-dev library. Therefore, I will be unable to build the numpy package because I do not have the required Python development headers.

This is how you reproduce the issue:

$ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.10.1.tar.gz
$ tar xzvf virtualenv-1.10.1.tar.gz
$ python virtualenv-1.10.1/virtualenv.py test
$ source test/bin/activate
$ pip install numpy

However, while testing pip with TUF, instead of seeing the appropriate failure message, we eventually see something like:

   out = check_types(*a, **kw)

  File "numpy/core/setup.py", line 271, in check_types

    "Cannot compile 'Python.h'. Perhaps you need to "\

SystemError: Cannot compile 'Python.h'. Perhaps you need to install python-dev|python-devel.

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/test/build/numpy
Traceback (most recent call last):
  File "/tmp/test/bin/pip", line 9, in <module>
    load_entry_point('pip==1.4.1', 'console_scripts', 'pip')()
  File "/tmp/test/local/lib/python2.7/site-packages/pip/__init__.py", line 148, in main
    return command.main(args[1:], options)
  File "/tmp/test/local/lib/python2.7/site-packages/pip/basecommand.py", line 169, in main
    text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 72: ordinal not in range(128)

Changing line 169 to this fixes the issue:

text = '\n'.join(s.decode('utf-8') for s in complete_log).encode('utf-8')

However, I am not sure that that is the appropriate fix. Furthermore, it probably does not port well into Python 3. It is probably best to fix the issue with a solution that works across all supported Python versions.

Issue Analytics

  • State:closed
  • Created 10 years ago
  • Comments:38 (19 by maintainers)

github_iconTop GitHub Comments

1reaction
neutronestcommented, Nov 13, 2014

Is it fixed already…?

0reactions
zhirzhcommented, Dec 28, 2014

hmm, this is strange. updating my system doesn’t do anything to the pip package, since the python-pip package itself is in version 1.5.4-1.

however, Installing pip from get-pip.py installed tha latest pip.

Problem solved.

Read more comments on GitHub >

github_iconTop Results From Across the Web

python - python3 UnicodeDecodeError while logging to console
I found the answer here Python3 UnicodeDecodeError. Let me expand: This is solved by setting the environment value LC_CTYPE to en_US.
Read more >
Python 3 Notes: Reading and Writing Methods
"UnicodeDecodeError" means you have a file encoding issue. Each computer has its own system-wide default encoding, and the file you are trying to...
Read more >
Unicode HOWTO — Python 3.11.1 documentation
Once you've written some code that works with Unicode data, the next problem is input/output. How do you get Unicode strings into your...
Read more >
Cannot figure out why this is giving a UnicodeDecodeError
The install appears to run successfully. I've tried it multiple ways and when I can finally get the install to succeed (the installer...
Read more >
UnicodeDecodeError while using LaTeX in Python - TeX
When trying to run a code containing text written for LaTeX I get an error. UnicodeDecodeError: 'charmap' codec can't decode byte 0xa1 in...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found