question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

`dvc import` from `s3://...` fails when s3 provided by 3rd party provider.

See original GitHub issue
  • Standard aws client works, if endpoint url provided.
    • aws --endpoint-url https://<my-endpoint> s3 ls <my-path>
  • DVC repository works, endpoint_url provided
['remote "minio"']
url = s3://<my-path>
endpointurl = <my-endpoint>

However dvc import fails because no capability to specify endpoint url More details: https://gist.github.com/bwalsh/1afea0a2499b5e81c507dbc24521038e

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Reactions:3
  • Comments:9 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
ghostcommented, Oct 31, 2018

@efiop, I’m going to label this as documentation to improve/clarify the remote:// “protocol”

1reaction
bwalshcommented, Oct 31, 2018

@efiop : yes I was aware. The semantics of dvc import are clear, it’s the semantics the of remote:// by import

Read more comments on GitHub >

github_iconTop Results From Across the Web

"dvc add -external S3://mybucket/data.csv" is failing with ...
You are using s3://dataset/ in your add --external command. I think that DVC tries to connect to a bucket called dataset and to...
Read more >
Metaflow/community - Gitter
However, I am getting a S3 datastore operation _get_s3_object failed (An error occurred (400) when calling the HeadObject operation: Bad Request) ...
Read more >
Resolve "Unable to locate credentials" error in ... - Amazon AWS
An "Unable to locate credentials" error indicates that Amazon S3 can't find the credentials to authenticate AWS API calls. To resolve this issue ......
Read more >
Resolve "Unable to locate credentials" error in Amazon S3
How do I fix this? Resolution. An "Unable to locate credentials" error indicates that Amazon S3 can't find the credentials to authenticate AWS...
Read more >
Data Version Control With Python and DVC - Real Python
But data version control, managing changes to models and datasets, isn't so well established. It's not easy to keep track of all the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found