Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

0.4.1 multiple calls to info() fail

See original GitHub issue

Just updated to 0.4.1 from 0.4.0 and it appears to have introduced a regression. I’ve reproduced it on python 3.8 and 3.6, on several different buckets. Am using fsspec==0.7.0

Test case:

import s3fs
import fsspec
fs = fsspec.filesystem('s3', config_kwargs={'region_name':'us-east-1'})
fs.info('s3://czi.cxg-public/cxg/') # works
fs.info('s3://czi.cxg-public/cxg/') # fails

Example result with 0.4.1:

# python
Python 3.8.2 (default, Feb 26 2020, 04:23:39) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import s3fs
>>> import fsspec
>>> fs = fsspec.filesystem('s3', config_kwargs={'region_name':'us-east-1'})
>>> fs.info('s3://czi.cxg-public/cxg/') # works
{'Key': 'czi.cxg-public/cxg', 'Size': 0, 'StorageClass': 'DIRECTORY', 'type': 'directory', 'size': 0, 'name': 'czi.cxg-public/cxg'}
>>> fs.info('s3://czi.cxg-public/cxg/') # fails
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/mnt/c/Users/bruce/projects/cellxgene/venv38/lib/python3.8/site-packages/s3fs/core.py", line 525, in info
    if self.version_aware or (key and self._ls_from_cache(path) is None) or refresh:
  File "/mnt/c/Users/bruce/projects/cellxgene/venv38/lib/python3.8/site-packages/fsspec/spec.py", line 313, in _ls_from_cache
    raise FileNotFoundError(path)
FileNotFoundError: s3://czi.cxg-public/cxg/

And on 0.4.0:

# python
Python 3.8.2 (default, Feb 26 2020, 04:23:39) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import s3fs
>>> import fsspec
>>> fs = fsspec.filesystem('s3', config_kwargs={'region_name':'us-east-1'})
>>> fs.info('s3://czi.cxg-public/cxg/') # works
{'Key': 'czi.cxg-public/cxg', 'Size': 0, 'StorageClass': 'DIRECTORY', 'type': 'directory', 'size': 0, 'name': 'czi.cxg-public/cxg'}
>>> fs.info('s3://czi.cxg-public/cxg/') # works
{'Key': 'czi.cxg-public/cxg', 'Size': 0, 'StorageClass': 'DIRECTORY', 'type': 'directory', 'size': 0, 'name': 'czi.cxg-public/cxg'}

Issue Analytics

State:
Created 3 years ago
Reactions:1
Comments:9 (9 by maintainers)

Top GitHub Comments

1reaction

bkmartinjrcommented, Mar 30, 2020

sure! Will need a day or two…

0reactions

bkmartinjrcommented, Apr 5, 2020

Hi @martindurant - looks like you beat me to it and have already fixed. I have confirmed that release 0.4.2 resolves the above test case.

I will still submit a PR for a test case. Thank you for the rapid fix!

Top Results From Across the Web

Too many open files error · Issue #11201 · pytorch ... - GitHub

I am using Ignite's launcher to start the distributed processes, so am not calling dist.init_process_group() myself directly. But I had not ...

Error: package or namespace load failed for 'tidyverse' in ...

hi, try to uninstall and reinstall the rlang package. Also check your libray paths. You can do so by typing .libPaths() . –...

Changelog - dplyr

The rows_*() functions now fail elegantly if y is a zero column data frame and ... Multiple calls to if_any() and/or if_all() in...

Bug listing with status RESOLVED with resolution FIXED as at ...

... Bug:200 - "memprof-0.4.1 dependency string format error" status:RESOLVED ... "sandbox addwrite() etc. funcs don't recognize paths with multiple slashes" ...

Colorama - PyPI

It's not safe to call init multiple times; you can end up with multiple layers ... To stop using Colorama before your program...