Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

isdir()/info() doesn't work properly for

See original GitHub issue

When an empty directory created (with a trailing slash), the initial info() call returns the object itself and also caches but the call after that returns it in the form of a directory instead of the just what it got from API call. Here is a simple reproducer (first isdir() returns False, second one returns True);

import secrets
from gcsfs import GCSFileSystem
from pprint import pprint

BUCKET = <FILL HERE>
BASE_PATH = BUCKET + "/" + secrets.token_hex(12)
EMPTY_DIR = BASE_PATH + "/empty_dir/"

fs = GCSFileSystem()
fs.makedirs(BASE_PATH)
fs.touch(EMPTY_DIR)

print("isdir()?", fs.isdir(EMPTY_DIR))
print("isdir() (cached)?", fs.isdir(EMPTY_DIR))

fs.invalidate_cache()
print("\nInitial call: ")
pprint(fs.info(EMPTY_DIR))
print("\nSecondary call (from cache): ")
pprint(fs.info(EMPTY_DIR))

isdir()?  False
isdir() (cached)? True

Initial call: 
{'bucket': 'batuhan-test',
 'contentType': 'application/octet-stream',
 'crc32c': 'AAAAAA==',
 'etag': 'COeb0Jbagu8CEAE=',
 'generation': '1614175995694567',
 'id': 'batuhan-test/39bd4b9d6b4a71c8fd857e5d/empty_dir//1614175995694567',
 'kind': 'storage#object',
 'md5Hash': '1B2M2Y8AsgTpgAmY7PhCfg==',
 'mediaLink': 'https://www.googleapis.com/download/storage/v1/b/batuhan-test/o/39bd4b9d6b4a71c8fd857e5d%2Fempty_dir%2F?generation=1614175995694567&alt=media',
 'metageneration': '1',
 'name': 'batuhan-test/39bd4b9d6b4a71c8fd857e5d/empty_dir/',
 'selfLink': 'https://www.googleapis.com/storage/v1/b/batuhan-test/o/39bd4b9d6b4a71c8fd857e5d%2Fempty_dir%2F',
 'size': 0,
 'storageClass': 'STANDARD',
 'timeCreated': '2021-02-24T14:13:15.696Z',
 'timeStorageClassUpdated': '2021-02-24T14:13:15.696Z',
 'type': 'file',
 'updated': '2021-02-24T14:13:15.696Z'}

Secondary call (from cache): 
{'bucket': 'batuhan-test',
 'name': 'batuhan-test/39bd4b9d6b4a71c8fd857e5d/empty_dir',
 'size': 0,
 'storageClass': 'DIRECTORY',
 'type': 'directory'}

Issue Analytics

State:
Created 3 years ago
Comments:8 (8 by maintainers)

Top GitHub Comments

1reaction

martindurantcommented, Feb 25, 2021

You will find similar discussions in other issues on this repo and elsewhere - and it does confuse people. I don’t know what or where we can write to clarify this. I mention this briefly in my webinar.

To be sure, the CLI only emulates directories in certain cases (thanks for the link) and the REST API and SKD clients do not so this at all. I would say that the console, with its “create folder” button is definitely the odd one out.

0reactions

isidenticalcommented, Mar 1, 2021

Looks like this is a duplicate of #312