Feature request: Use adlfs to access public blobs

Originally posted it at https://stackoverflow.com/questions/63856476/adlfs-create-azureblobfilesystem-with-credential-none but should probably post here. Need rep points to create a adlfs tag and fsspec tags…

I’m going through this: https://azure.microsoft.com/en-us/services/open-datasets/catalog/goes-16/

and I was curious how to do the equivalent of s3fs.S3FileSystem(anon=True) with adlfs.AzureBlobFileSystem.

It seems credential=None is a good candidate (https://docs.microsoft.com/en-us/python/api/azure-storage-blob/azure.storage.blob.containerclient?view=azure-python) but as far as i’m aware it hasn’t been implemented?

>>> import adlfs
>>> fs = adlfs.AzureBlobFileSystem(credential=None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ray/local/bin/anaconda3/envs/test_env/lib/python3.8/site-packages/fsspec/spec.py", line 52, in __call__
    obj = super().__call__(*args, **kwargs)
TypeError: __init__() missing 1 required positional argument: 'account_name'

[hayesgb]

I’ve added anonymous authentication to AzureBlobFileSystem with #106 and #107 so the above goes example works as follows:

storage_options = {'account_name': 'goes'}
fs = AzureBlobFileSystem(**storage_options)
fs.ls("noaa-goes16")

An example Jupyter Notebook on Azure can be found here. The above doesn’t work if you just try fs.ls("/").

I believe @raybellwaves is right – the trick is making sure you get the account_name and container right.

BTW – The GOES data is stored in xarray, so I didn’t try opening the file.

Also successfully accessed the data here as follows:

storage_options={"account_name": "hlssa"}
fs = AzureBlobFileSystem(**storage_options)
fs.ls("hls")
ddf = dd.read_csv("abfs://hls/S2_TilingSystem2-1.txt", storage_options=storage_options)

Both .head() and .tail() work tag: @lostmygithubaccount

[hayesgb]

I’m trying to get to releases on the first week of the month. You can install what’s in the master branch directly with pip from github if you need it now.

On Sep 24, 2020, at 12:28 PM, Cody [e-mail] wrote:

will this be published to pypi soon @hayesgb ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.