Failed uploading: cannot import name 'BlockType' from 'azure.storage.blob'

Greetings ClearML team!

We have an issue with the latest ClearML version including PR https://github.com/allegroai/clearml/pull/548 (support newer azure storage python version).

Affected versions: 1.2.0, 1.3.0

Some uploads are failing like this:

2022-03-05 17:32:38,194 - clearml.storage - ERROR - Failed uploading: cannot import name 'BlockType' from 'azure.storage.blob' (/root/.clearml/venvs-builds/3.8/lib/python3.8/site-packages/azure/storage/blob/__init__.py)
2022-03-05 17:32:38,194 - clearml.storage - ERROR - Exception encountered while uploading Upload failed
Traceback (most recent call last):
  File "/root/.clearml/venvs-builds/3.8/task_repository/repo.git/export/model_export.py", line 42, in default_clearml_export
    task.upload_artifact(
  File "/root/.clearml/venvs-builds/3.8/lib/python3.8/site-packages/clearml/task.py", line 1699, in upload_artifact
    return self._artifacts_manager.upload_artifact(
  File "/root/.clearml/venvs-builds/3.8/lib/python3.8/site-packages/clearml/binding/artifacts.py", line 617, in upload_artifact
    uri = self._upload_local_file(local_filename, name,
  File "/root/.clearml/venvs-builds/3.8/lib/python3.8/site-packages/clearml/binding/artifacts.py", line 799, in _upload_local_file
    StorageManager.upload_file(local_file.as_posix(), uri, wait_for_upload=True, retries=ev.retries)
  File "/root/.clearml/venvs-builds/3.8/lib/python3.8/site-packages/clearml/storage/manager.py", line 76, in upload_file
    return CacheManager.get_cache_manager().upload_file(
  File "/root/.clearml/venvs-builds/3.8/lib/python3.8/site-packages/clearml/storage/cache.py", line 91, in upload_file
    result = helper.upload(
  File "/root/.clearml/venvs-builds/3.8/lib/python3.8/site-packages/clearml/storage/helper.py", line 579, in upload
    res = self._do_upload(src_path, dest_path, extra, cb, verbose=False, retries=retries)
  File "/root/.clearml/venvs-builds/3.8/lib/python3.8/site-packages/clearml/storage/helper.py", line 993, in _do_upload
    raise last_ex
ValueError: Upload failed

The problematic line is in helper.py:

from azure.storage.blob import BlockType  # noqa
client.append_block(chunk, BlockType.AppendBlob)

I have tested several versions of azure-blob-service (12.0.0, 12.1.0, 12.2.0, 12.3.0, 12.4.0, 12.5.0, 12.6.0, 12.7.0, 12.8.0, 12.9.0) and with none of them, the import works.

Which azure-storage version this is supposed to run with?

In the example code from Azure I found that appending to blob works differently: https://github.com/Azure/azure-storage-python/blob/4306898850dd21617644fc537a57d025e833db74/samples/blob/append_blob_usage.py#L248

Thanks for you help!

[jkhenning]

Hi @tkupek,

Thanks for this detailed description! We’ll update soon 🙂

[tkupek]

Thank you for your patience. We upgraded to 1.3.1 and the issue is fully resolved. Great work!