[BUG] Credentials are set, but tracking server lacks permission to list artifacts under the current s3 artifact directory

Willingness to contribute

Yes. I would be willing to contribute a fix for this bug with guidance from the MLflow community.

MLflow version

1.29

System information

Linux Ubuntu MLFlow v1.29.0

Describe the problem

Unable to list artifacts stored under s3://my-s3-bucket/1/5e295e5a735949a98588637cc3735ac4/artifacts for the current run. Please contact your tracking server administrator to notify them of this error, which can happen when the tracking server lacks permission to list artifacts under the current run’s root artifact directory.

But the notebook says:

Registered model 'model-v' already exists. Creating a new version of this model...
2022/09/25 21:24:54 INFO mlflow.tracking._model_registry.client: Waiting up to 300 seconds for model version to finish creation.                     Model name: model-v, version 9 Created version '9' of model 'adept_model-v'.

Tracking information

helm install mlflow /helm_charts/mlflow --kubeconfig=my_kubeconfig --set backendStore.filepath=sqlite:///mnt/mlflow.db --set AWS_DEFAULT_REGION=us-east-1 --set AWS_SECRET_ACCESS_KEY=XXXX --set AWS_ACCESS_KEY_ID=XXXX --set defaultArtifactRoot=s3://my-s3-bucket

Code to reproduce issue

there is no code its a helm chart and the config isnt connecting to s3

Stack trace

helm install mlflow /helm_charts/mlflow --kubeconfig=my_kubeconfig --set backendStore.filepath=sqlite:///mnt/mlflow.db --set AWS_DEFAULT_REGION=us-east-1 --set AWS_SECRET_ACCESS_KEY=XXXX --set AWS_ACCESS_KEY_ID=XXXX --set defaultArtifactRoot=s3://my-s3-bucket

Other info / logs

helm install mlflow /helm_charts/mlflow --kubeconfig=my_kubeconfig --set backendStore.filepath=sqlite:///mnt/mlflow.db --set AWS_DEFAULT_REGION=us-east-1 --set AWS_SECRET_ACCESS_KEY=XXXX --set AWS_ACCESS_KEY_ID=XXXX --set defaultArtifactRoot=s3://my-s3-bucket

What component(s) does this bug affect?

• area/artifacts: Artifact stores and artifact logging
• area/build: Build and test infrastructure for MLflow
• area/docs: MLflow documentation pages
• area/examples: Example code
• area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry
• area/models: MLmodel format, model serialization/deserialization, flavors
• area/pipelines: Pipelines, Pipeline APIs, Pipeline configs, Pipeline Templates
• area/projects: MLproject format, project running backends
• area/scoring: MLflow Model server, model deployment tools, Spark UDFs
• area/server-infra: MLflow Tracking server backend
• area/tracking: Tracking Service, tracking client APIs, autologging

What interface(s) does this bug affect?

• area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev server
• area/docker: Docker use across MLflow’s components, such as MLflow Projects and MLflow Models
• area/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registry
• area/windows: Windows support

What language(s) does this bug affect?

• language/r: R APIs and clients
• language/java: Java APIs and clients
• language/new: Proposals for new client languages

What integration(s) does this bug affect?

• integrations/azure: Azure and Azure ML integrations
• integrations/sagemaker: SageMaker integrations
• integrations/databricks: Databricks integrations

[salinaaaaaa]

im working on a helm chart you guys can take and refactor – coming soon, sit tight!!

On Sun, Oct 2, 2022 at 7:28 PM mlflow-automation @.***> wrote:

@BenWilson2 https://github.com/BenWilson2 @dbczumar https://github.com/dbczumar @harupy https://github.com/harupy @WeichenXu123 https://github.com/WeichenXu123 Please assign a maintainer and start triaging this issue.

— Reply to this email directly, view it on GitHub https://github.com/mlflow/mlflow/issues/6882#issuecomment-1264777340, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHVQHCRTDELQSSGUUJSBRLWBISD3ANCNFSM6AAAAAAQVIEKFU . You are receiving this because you authored the thread.Message ID: @.***>

[mlflow-automation]

@BenWilson2 Please reply to comments.