Bug: zarr.open behaves different than zarr.open_group with mode `w-` on `gs://` URIs

The convenient function zarr.open does not work to create a zarr group on a gcp bucket with mode w-:

Minimal, reproducible code sample:

an_empty_gs_uri = "gs://my-bucket/test.zarr"
zarr.open(an_empty_gs_uri, mode=w-)

Problem description

If we use w- mode above code breaks, even if we dont have a group at the URI given:

---------------------------------------------------------------------------
ReadOnlyError                             Traceback (most recent call last)
<ipython-input-11-d0b46f9c7e3f> in <module>
----> 1 zarr.open(t3, mode='w-')

~/merantix/mxlabs-chameleon/venv/lib/python3.8/site-packages/zarr/convenience.py in open(store, mode, **kwargs)
     84             return open_array(store, mode=mode, **kwargs)
     85         else:
---> 86             return open_group(store, mode=mode, **kwargs)
     87 
     88     elif mode == "a":

~/merantix/mxlabs-chameleon/venv/lib/python3.8/site-packages/zarr/hierarchy.py in open_group(store, mode, cache_attrs, synchronizer, path, chunk_store, storage_options)
   1181             raise ContainsGroupError(path)
   1182         else:
-> 1183             init_group(store, path=path, chunk_store=chunk_store)
   1184 
   1185     # determine read only status

~/merantix/mxlabs-chameleon/venv/lib/python3.8/site-packages/zarr/storage.py in init_group(store, overwrite, path, chunk_store)
    470 
    471     # initialise metadata
--> 472     _init_group_metadata(store=store, overwrite=overwrite, path=path,
    473                          chunk_store=chunk_store)
    474 

~/merantix/mxlabs-chameleon/venv/lib/python3.8/site-packages/zarr/storage.py in _init_group_metadata(store, overwrite, path, chunk_store)
    497     meta = dict()  # type: ignore
    498     key = _path_to_prefix(path) + group_meta_key
--> 499     store[key] = encode_group_metadata(meta)
    500 
    501 

~/merantix/mxlabs-chameleon/venv/lib/python3.8/site-packages/zarr/storage.py in __setitem__(self, key, value)
   1058     def __setitem__(self, key, value):
   1059         if self.mode == 'r':
-> 1060             raise ReadOnlyError()
   1061         key = self._normalize_key(key)
   1062         path = self.dir_path(key)

ReadOnlyError: object is read-only

If we use mode w it works. Also zarr.open_group works with the URI and the mode w- . From having a quick look, the underlying fsstore seems to be configured in read only mode when we use zarr.open(gs_uri, mode=w-).

Version and installation information

Please provide the following:

• Value of zarr.__version__ 2.6.1
• Value of numcodecs.__version__ 0.7.3
• Value of fsspec.__version__ 0.8.7
• Version of Python interpreter 3.8.5
• Operating system (Linux/Windows/Mac): Linux
• How Zarr was installed (e.g., “using pip into virtual environment”, or “using conda”) pip

[arogozhnikov]

@joshmoore I confirm that with 2.11.1 the issue is solved! 🎉

[arogozhnikov]

@joshmoore I’m just exploring/testing format and did not use it previously.