StorageManager cannot work with MinIO

Hi,

Describe the bug

i want to store all task, model, dataset, etc to minio. I see on document, StorageManager to manage upload, download data to our storage platform (i use minio). this is my config:

    aws {
        s3 {
            # use_credentials_chain: false
            region: "my_xx"
            key: "my_xx"
            secret: "my_xx"
            credentials: [
                {
                    # This will apply to all buckets in this host (unless key/value is specifically provided for a given bucket)
                    host: "minio_ip:9000"
                    key: "my_xx"
                    secret: "my_xx"
                    region: "my_xx"
                    multipart: false
                    secure: false
                }
            ]
        }
    }

To reproduce

i run this code to try upload but not works.

from clearml import StorageManager

manager = StorageManager()
manager.upload_folder(
    local_folder="/home/****/project1/dataset", 
    remote_url="s3://ip_minio:9000/test-bucket/dateset-clearml"
)

and return the error

...
Failed uploading: Could not connect to the endpoint URL: "http://s3.region-name.amazonaws.com/test-bucket/dateset-clearml/lq/seg_151a8cb2b37f4de6954aaf9fd6e0ee48_1.jpg"
Failed uploading: Could not connect to the endpoint URL: "http://s3.region-name.amazonaws.com/test-bucket/dateset-clearml/lq/seg_9f76595542e342c4a61f83a4b11c9e11_2.jpg"
...

the url seems still using aws, not minio_ip. is there i miss for config in MinIO?

Environment

• Server : self hosted
• ClearML SDK Version 1.6.4
• ClearML Server Version: WebApp: 1.6.0-213 • Server: 1.6.0-213 • API: 2.20
• Python Version 3.10.4
• OS Linux

Thanks.

[erezalg]

Hi @muhammadAgfian96,

can you try with latest released RC and let us know if the issue is resolved?

[erezalg]

Hi @muhammadAgfian96,

Sorry for the late reply, this is planned to be released in the next version. We should update here once it’s out.