Calling AzureBlobFileSystem.cat on a file path, adds a "/" to the end of the file path

I was using dask + zarr to store arrays on azure blob storage using this library, but I ran into an issue.

What happens is that:

1. At some point when loading a zarr file from the storage fsspec.mapping.FSMap is created (with .fs being an AzureBlobFileSystem instance);
2. It calls self.fs.cat(k) where k is a string representing a file path on the blob storage: e.g. my-blob/my-array/.zarray; and self.fs is the AzureBlobFileSystem instance;
3. AzureBlobFileSystem.cat calls AzureBlobFileSystem._expand_path at some point which runs this line:

https://github.com/dask/adlfs/blob/3874b3e536fe6b24c824ee096566c8620b623dfa/adlfs/spec.py#L1351

1. The thing returns: my-blob/my-array/.zarray/ and later on the loading crashes with the following error:

ResourceNotFoundError: Operation returned an invalid status 'The specified blob does not exist.'
ErrorCode:BlobNotFound

---

I think the solution is just to make sure we don’t add a “/” in _expand_path

[hayesgb]

Fixed with #217

[TomAugspurger]

Thanks! I do think it’s worth having a longer discussion around the ideal behavior here, about how to consistently handle pseduo-directories in these object stores. Maybe that discussion has happened though, I haven’t followed closely (edit: that’s happening in https://github.com/intake/filesystem_spec/issues/562).

I haven’t tested it yet, but something like this might work

diff --git a/adlfs/spec.py b/adlfs/spec.py
index 298fa7b..b13bcfd 100644
--- a/adlfs/spec.py
+++ b/adlfs/spec.py
@@ -1263,6 +1263,11 @@ class AzureBlobFileSystem(AsyncFileSystem):
 
         async with self.service_client.get_blob_client(container_name, path) as bc:
             exists = await bc.exists()
+
+        if path.endswith("/") and not exists:
+            async with self.service_client.get_blob_client(container_name, path[:-1]) as bc:
+                exists = await bc.exists()
+
         return exists
 
     async def _pipe_file(self, path, value, overwrite=True, **kwargs):

That solves the exists() side. Unfortunately things like .open() will need to be updated as well…