question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

missing information about s3 usage

See original GitHub issue

Docs don’t explicitly say that when one is trying to use s3 or s3 compatible storage there should already exist bucket.

Then, if you would try to push data to s3 and bucket wouldn’t exist, you would get the following:

Preparing to upload data to 's3://my/remote/'
Preparing to collect status from s3://my/remote/
[############                  ] 40% Collecting information
ERROR: failed to push data to the cloud - An error occurred (NoSuchBucket) when calling the ListObjectsV2 operation: Unknown

So I propose to explicitly say in docs that you should already have existing bucket to be able to push to s3. Following docs are related to s3 usage: https://dvc.org/doc/get-started/configure https://dvc.org/doc/commands-reference/remote-add Related discussion in ods: https://opendatascience.slack.com/archives/CGGLZJ119/p1559831715009900

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Reactions:2
  • Comments:9 (9 by maintainers)

github_iconTop GitHub Comments

1reaction
shchekleincommented, Jun 22, 2019

@Naba7 definitely, let’s mention this!

1reaction
dnabanita7commented, Jun 22, 2019

Actually, it is mentioned in use-cases to create a bucket before connecting it to dvc remote as of here https://dvc.org/doc/use-cases/share-data-and-model-files . But I think mentioning it in commands-reference/remote and commands-reference/config will do good.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Troubleshoot an AWS DMS task that is successful, but no data ...
This error occurs when the AWS Identity and Access Management (IAM) role for the Amazon S3 source endpoint doesn't have the correct permissions: ......
Read more >
S3 and missing rows in Redshift - Medium
We will deduct information about previous row if it is missing. ... We use a custom time interval to check rows (for example...
Read more >
Identifying missing files in S3 that are not downloaded from FTP
You can abort the listing at the moment you encounter the first file, which is already at the S3. Of course that's only...
Read more >
10 things you should know about using AWS S3 - Sumo Logic
Learn how to optimize Amazon S3 with top tips and best practices. Bucket limits, transfer speeds, storage costs, and more – get answers...
Read more >
AWS S3 CloudWatch Metrics not appearing and missing
I've double-checked and everything else appears to be working correctly, but the free 24 hour S3 data doesn't appear in our metrics.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found