read_csv from HTTPs + basic-auth + custom port throws an error (urlopen error)
See original GitHub issueCode Sample, a copy-pastable example if possible
import pandas as pd
df = pd.read_csv('https://user-name:pwd@my.fqdn.com:8080/get_content.csv')
# fails with urlopen error [Errno 11003] getaddrinfo failed Python2.7 as well as Python 3.x
Problem description
HTTPS basic auth is very common. This URL format works in Excel, other text editors, etc. This url works in requests
library. It seems like the scenario doesnt work because the underlying urlopen
doesnt work
# I think it fails because the underlying code here fails in pandas/io/common.py:
from urllib2 import urlopen as _urlopen
b = _urlopen('https://un-name:pwd@my.fqdn.com:8080/get_content.csv')
# fails with urlopen error [Errno 11003] getaddrinfo failed
see stackoverflow issue urllib basic auth
Only way to overcome this is using requests
+ StringIO
?
Expected Output
be able to get a CSV loaded dataframe
Output of pd.show_versions()
python: 2.7.13.final.0
python-bits: 64
pandas: 0.20.1
Issue Analytics
- State:
- Created 6 years ago
- Reactions:1
- Comments:39 (24 by maintainers)
Top Results From Across the Web
Handling HTTP authentication when accesing remote urls via ...
Python code to read it via pandas.read_csv would look like the following. ... print(ex) HTTP Error 401: Unauthorized In [3]: from base64 import...
Read more >Requests Documentation - Read the Docs
Fixed bug with unintended Authorization header stripping for redirects using default ports (http/80, https/443). 3.9.14 2.20.0 (2018-10-18).
Read more >Getting Error 401 when calling the REST API from a different ...
It works for few build ids and then it starts throwing http error 401 basic auth failed. I have tried everything possible by...
Read more >Error while loading csv file - Python - The freeCodeCamp Forum
urlpath = 'https://raw.githubusercontent.com/datasets/gdp/master/data/gdp.csv' pd.read_csv(urlpath) URLError: <urlopen error [Errno 11004] ...
Read more >HTTP 401 Unauthorized Error | What Is and How to Fix?
A network 401 Error signifies that the resource is barred and needs authentication that the client did not provide. Just like many HTTP...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
This alone isn’t worth adding it as a dependency I think.
There is also same request for read_json https://github.com/pandas-dev/pandas/issues/10526