fetch_openml() does not use the local cache

Description

fetch_openml() does not use the local data cached in ~/scikit_learn_data.

Steps/Code to Reproduce

Run the following code twice. The first time, with a working Internet connexion, the second time without an Internet connexion.

from sklearn.datasets import fetch_openml
mnist = fetch_openml("mnist_784", version=1)
print(mnist.data.shape)

Expected Results

First run (with Internet access): (70000, 784)

Second run (without Internet access): (70000, 784)

Actual Results

First run (with Internet access): (70000, 784) Second run (without Internet access):

Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1392, in connect
    super().connect()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 936, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socket.py", line 704, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socket.py", line 745, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 8] nodename nor servname provided, or not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/ageron/.virtualenvs/ml/lib/python3.6/site-packages/sklearn/datasets/openml.py", line 466, in fetch_openml
    data_info = _get_data_info_by_name(name, version, data_home)
  File "/Users/ageron/.virtualenvs/ml/lib/python3.6/site-packages/sklearn/datasets/openml.py", line 275, in _get_data_info_by_name
    data_home)
  File "/Users/ageron/.virtualenvs/ml/lib/python3.6/site-packages/sklearn/datasets/openml.py", line 119, in _get_json_content_from_openml_api
    response = _open_openml_url(url, data_home)
  File "/Users/ageron/.virtualenvs/ml/lib/python3.6/site-packages/sklearn/datasets/openml.py", line 55, in _open_openml_url
    fsrc = urlopen(req)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 8] nodename nor servname provided, or not known>

AFAICT, the data is correctly cached in the ~/scikit_learn_data directory, but simply ignored by fetch_openml():

$ tree ~/scikit_learn_data/openml/
/Users/ageron/scikit_learn_data/openml/
└── openml.org
    ├── api
    │   └── v1
    │       └── json
    │           └── data
    │               ├── 554.gz
    │               ├── features
    │               │   └── 554.gz
    │               └── list
    │                   └── data_name
    │                       └── mnist_784
    │                           └── limit
    │                               └── 2
    │                                   └── data_version
    │                                       └── 1.gz
    └── data
        └── v1
            └── download
                └── 52667.gz

15 directories, 4 files

Versions

Darwin-17.7.0-x86_64-i386-64bit Python 3.6.6 (default, Jun 28 2018, 05:43:53) [GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2)] NumPy 1.15.2 SciPy 1.1.0 Scikit-Learn 0.20.0

Same results as well on:

Linux-4.15.0-36-generic-x86_64-with-Ubuntu-18.04-bionic Python 3.6.5 (default, Apr 1 2018, 05:46:30) [GCC 7.3.0] NumPy 1.15.1 SciPy 1.1.0 Scikit-Learn 0.20.0

[elissavetv]

Hello, I am getting the exact same error as above, is there a fix? Many thanks.

[shasherazi]

Here’s a workaround tho