Unable to install package with custom pypi source

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: macOS 10.14.6 / docker image python:3.5
• Poetry version: 1.0.0
• Link of a Gist repository with the contents of your pyproject.toml file: https://github.com/ocavue/poetry_issue_celery

Issue

The steps to reproduce this bug is in the readme.

In short, I can’t install any package with a custom pypi source (https://pypi.douban.com/simple/). pip3 install celery -i https://pypi.douban.com/simple/ works find, so this should be a bug of poetry.

[ocavue]

@thomaszdxsn

I have already found the reason of this issue as my comments above. We need not more discussion about the reason.

If you just want to avoid this issue, your can use https://mirrors.aliyun.com/pypi/simple.

[ocavue]

I did some little research. poetry use requests.get() to fetch packages from PyPI repository. request.get() will follow 301 redirection by default. The reason that poetry can’t install packages from https://pypi.douban.com/ is that Douban server return 418 I’m a teapot. I guess that this is an anti-scraping logic.

Use curl to visit pypi.douban.com. Get 301:

$ curl -v 'https://pypi.douban.com/simple/celery/'
...
< HTTP/1.1 301 Moved Permanently
...

Use requests to visit pypi.douban.com. Get 418:

In [1]: import requests

In [2]: requests.get('https://pypi.douban.com/simple/celery/')
Out[2]: <Response [418]>

In [3]:  requests.get('https://pypi.doubanio.com/simple/celery/')
Out[3]: <Response [200]>

In [4]: requests.get('https://pypi.douban.com/simple/celery/', headers={'user-agent': 'poetry'})
Out[4]: <Response [200]>

To avoid this 418 response, we can simply pass a user-agent into the request header.

Since adding a header in the request can fix this issue and I couldn’t see any disadvantage about this change, I can submit a PR to do it if you agree with me. @sdispater

---

Douban is one of the biggest Chinese companies that use Python as their main back-end language. The PyPI mirror their offer is very famous in the Chinese Python developer community.