UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position 117: ordinal not in range(128)

Description

I really don’t know what’s wrong. But this starts happening after 21.2.1 release (https://github.com/aminvakil/docker-ansible/runs/3153704426?check_suite_focus=true) and worked just fine until 21.1.3 release (https://github.com/aminvakil/docker-ansible/runs/3148228248?check_suite_focus=true).

This happens on pip3 install ansible command system-wide. This is running on (centos8, centos7, debian10, debian11, ubuntu20.04, ubuntu18.04, fedora34, fedora33, fedora32) and only broke in centos7 and ubuntu18.04 (Please kindly see https://github.com/aminvakil/docker-ansible/runs/3148228248?check_suite_focus=true).

---

Current workaround: Ubuntu 18.04: https://github.com/pypa/pip/issues/10219#issuecomment-887337037 CentOS 7: https://github.com/pypa/pip/issues/10219#issuecomment-888127061

---

Expected behavior

Ansible gets installed.

pip version

21.2.1

Python version

3.6.7

OS

Ubuntu 18.04

How to Reproduce

Try building this dockerfile:

FROM ubuntu:bionic

LABEL maintainer="Amin Vakil <[e-mail]>"

RUN apt-get update \
    && apt-get install -y --no-install-recommends \
       python3-setuptools \
       python3-pip \
       python3-wheel \
       systemd \
       sudo \
    && rm -Rf /var/lib/apt/lists/* \
    && rm -Rf /usr/share/doc && rm -Rf /usr/share/man \
    && apt-get clean

RUN pip3 install --upgrade pip && pip3 install ansible

ENTRYPOINT ["/bin/systemd"]

or

FROM centos:7

LABEL maintainer="Amin Vakil <[e-mail]>"

ENV container docker
RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == \
systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;

RUN yum -y install python3-pip sudo && yum clean all

RUN pip3 install --upgrade pip && pip3 install ansible

VOLUME ["/sys/fs/cgroup"]

CMD ["/usr/sbin/init"]

Output

CentOS 7:

Step 6/9 : RUN pip3 install git+https://github.com/uranusjr/pip@locations-linux-system
 ---> Running in d8cf176851cb
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting git+https://github.com/uranusjr/pip@locations-linux-system
  Cloning https://github.com/uranusjr/pip (to locations-linux-system) to /tmp/pip-zfmsy5d7-build
Installing collected packages: pip
  Running setup.py install for pip: started
    Running setup.py install for pip: finished with status 'done'
Successfully installed pip-21.3.dev0
Removing intermediate container d8cf176851cb
 ---> b135aa678bda
Step 7/9 : RUN pip3 install ansible
 ---> Running in 9ce8bf831db5
Collecting ansible
  Downloading ansible-4.3.0.tar.gz (35.1 MB)
Collecting ansible-core<2.12,>=2.11.3
  Downloading ansible-core-2.11.3.tar.gz (6.8 MB)
ERROR: Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 173, in _main
    status = self.run(options, args)
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/cli/req_command.py", line 203, in wrapper
    return func(self, options, args)
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 316, in run
    reqs, check_supported_wheels=not options.target_dir
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
    collected.requirements, max_rounds=try_to_avoid_resolution_too_deep
  File "/usr/local/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 472, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/usr/local/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 366, in resolve
    failure_causes = self._attempt_to_pin_criterion(name)
  File "/usr/local/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 212, in _attempt_to_pin_criterion
    criteria = self._get_updated_criteria(candidate)
  File "/usr/local/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 203, in _get_updated_criteria
    self._add_to_criteria(criteria, requirement, parent=candidate)
  File "/usr/local/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 172, in _add_to_criteria
    if not criterion.candidates:
  File "/usr/local/lib/python3.6/site-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__
    return bool(self._sequence)
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 140, in __bool__
    return any(self)
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 128, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 32, in _iter_built
    candidate = func()
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 209, in _make_candidate_from_link
    version=version,
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 301, in __init__
    version=version,
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
    self.dist = self._prepare()
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 227, in _prepare
    dist = self._prepare_distribution()
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 306, in _prepare_distribution
    self._ireq, parallel_builds=True
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 508, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 552, in _prepare_linked_requirement
    self.download_dir, hashes
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 249, in unpack_url
    unpack_file(file.path, location, file.content_type)
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/utils/unpacking.py", line 256, in unpack_file
    untar_file(filename, location)
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/utils/unpacking.py", line 226, in untar_file
    with open(path, "wb") as destfp:
UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position 117: ordinal not in range(128)
The command '/bin/sh -c pip3 install ansible' returned a non-zero code: 2

Ubuntu 18.04:

Step 4/6 : RUN pip3 install git+https://github.com/uranusjr/pip@locations-linux-system
 ---> Running in 2e8524091c5a
Collecting git+https://github.com/uranusjr/pip@locations-linux-system
  Cloning https://github.com/uranusjr/pip (to locations-linux-system) to /tmp/pip-o235zpbf-build
Installing collected packages: pip
  Found existing installation: pip 9.0.1
    Not uninstalling pip at /usr/lib/python3/dist-packages, outside environment /usr
  Running setup.py install for pip: started
    Running setup.py install for pip: finished with status 'done'
Successfully installed pip-21.3.dev0
Removing intermediate container 2e8524091c5a
 ---> 2b5d0d0702ea
Step 5/6 : RUN pip3 install ansible
 ---> Running in 4419ef47a4eb
Collecting ansible
  Downloading ansible-4.3.0.tar.gz (35.1 MB)
Collecting ansible-core<2.12,>=2.11.3
  Downloading ansible-core-2.11.3.tar.gz (6.8 MB)
ERROR: Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/cli/base_command.py", line 173, in _main
    status = self.run(options, args)
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/cli/req_command.py", line 203, in wrapper
    return func(self, options, args)
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/commands/install.py", line 316, in run
    reqs, check_supported_wheels=not options.target_dir
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
    collected.requirements, max_rounds=try_to_avoid_resolution_too_deep
  File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/resolvelib/resolvers.py", line 472, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/resolvelib/resolvers.py", line 366, in resolve
    failure_causes = self._attempt_to_pin_criterion(name)
  File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/resolvelib/resolvers.py", line 212, in _attempt_to_pin_criterion
    criteria = self._get_updated_criteria(candidate)
  File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/resolvelib/resolvers.py", line 203, in _get_updated_criteria
    self._add_to_criteria(criteria, requirement, parent=candidate)
  File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/resolvelib/resolvers.py", line 172, in _add_to_criteria
    if not criterion.candidates:
  File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__
    return bool(self._sequence)
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 140, in __bool__
    return any(self)
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 128, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 32, in _iter_built
    candidate = func()
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/resolution/resolvelib/factory.py", line 209, in _make_candidate_from_link
    version=version,
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 301, in __init__
    version=version,
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
    self.dist = self._prepare()
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 227, in _prepare
    dist = self._prepare_distribution()
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 306, in _prepare_distribution
    self._ireq, parallel_builds=True
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/operations/prepare.py", line 508, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/operations/prepare.py", line 552, in _prepare_linked_requirement
    self.download_dir, hashes
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/operations/prepare.py", line 249, in unpack_url
    unpack_file(file.path, location, file.content_type)
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/utils/unpacking.py", line 256, in unpack_file
    untar_file(filename, location)
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/utils/unpacking.py", line 226, in untar_file
    with open(path, "wb") as destfp:
UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position 117: ordinal not in range(128)
The command '/bin/sh -c pip3 install ansible' returned a non-zero code: 2

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[aminvakil]

I can confirm adding

ENV LANG C.UTF-8
ENV LC_ALL C.UTF-8

fixes Ubuntu 18 Dockerfile.

For CentOS 7 this worked: https://github.com/pypa/pip/issues/10219#issuecomment-888127061

[uranusjr]

@hswong3i @rayfordj Please do not mention issues in commit messages, especially from someone else’s repository (or if you do, at least please don’t rebase it). Everyone in this thread gets notified whenever you add a commit (see the long list of back reference above?) and it is extremely annoying.