question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

read_csv from HTTPs + basic-auth + custom port throws an error (urlopen error)

See original GitHub issue

Code Sample, a copy-pastable example if possible

import pandas as pd
df = pd.read_csv('https://user-name:pwd@my.fqdn.com:8080/get_content.csv')
# fails with urlopen error [Errno 11003] getaddrinfo failed Python2.7 as well as Python 3.x

Problem description

HTTPS basic auth is very common. This URL format works in Excel, other text editors, etc. This url works in requests library. It seems like the scenario doesnt work because the underlying urlopen doesnt work

# I think it fails because the underlying code here fails in pandas/io/common.py:
from urllib2 import urlopen as _urlopen
b = _urlopen('https://un-name:pwd@my.fqdn.com:8080/get_content.csv')
# fails with urlopen error [Errno 11003] getaddrinfo failed

see stackoverflow issue urllib basic auth Only way to overcome this is using requests + StringIO ?

Expected Output

be able to get a CSV loaded dataframe

Output of pd.show_versions()

python: 2.7.13.final.0 python-bits: 64 pandas: 0.20.1

Issue Analytics

  • State:open
  • Created 6 years ago
  • Reactions:1
  • Comments:39 (24 by maintainers)

github_iconTop GitHub Comments

2reactions
TomAugspurgercommented, Jun 19, 2017

Why can’t we use requests ?

This alone isn’t worth adding it as a dependency I think.

0reactions
ron819commented, Aug 6, 2018

There is also same request for read_json https://github.com/pandas-dev/pandas/issues/10526

Read more comments on GitHub >

github_iconTop Results From Across the Web

Handling HTTP authentication when accesing remote urls via ...
Python code to read it via pandas.read_csv would look like the following. ... print(ex) HTTP Error 401: Unauthorized In [3]: from base64 import...
Read more >
Requests Documentation - Read the Docs
Fixed bug with unintended Authorization header stripping for redirects using default ports (http/80, https/443). 3.9.14 2.20.0 (2018-10-18).
Read more >
Getting Error 401 when calling the REST API from a different ...
It works for few build ids and then it starts throwing http error 401 basic auth failed. I have tried everything possible by...
Read more >
Error while loading csv file - Python - The freeCodeCamp Forum
urlpath = 'https://raw.githubusercontent.com/datasets/gdp/master/data/gdp.csv' pd.read_csv(urlpath) URLError: <urlopen error [Errno 11004] ...
Read more >
HTTP 401 Unauthorized Error | What Is and How to Fix?
A network 401 Error signifies that the resource is barred and needs authentication that the client did not provide. Just like many HTTP...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found