Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Use ~/.s3cfg for non AWS S3

See original GitHub issue

Feature Request

The documentation only shows AWS style configuration for S3 upload/download and doesn’t appear to accept other methods.

Desired behaviour

It would be desirable to use the ~/.s3cfg supported by tools like s3cmd.

[default]
access_key = XXXXXXXXXXXXXXXXXXX
encrypt = False
host_base = radosgw.XXX.XX.XX
host_bucket = %(bucket)s.radosgw.XXX.XX.XX
progress_meter = False
secret_key = XXXXXXXXXXXXXXXXXXXXXXXXXXXX
use_https = True

Actual behaviour

com.amazonaws.AmazonClientException: Unable to load AWS credentials from any provider in the chain

Steps to reproduce the behaviour

Use a non-AWS S3 implementation with a ~/.s3cfg instead of ~/.aws/config and ~/.aws/credentials

Environment (Browser or OS and Dockstore version)

Dockstore 1.1 CLI.

Issue Analytics

State:
Created 7 years ago
Comments:10 (6 by maintainers)

Top GitHub Comments

1reaction

keiranmrainecommented, Jan 23, 2017

@denis-yuen , all is well. There are secondary files for the BAM, the error message could do with indicating which file it is trying to pull as the output I provided was from a --debug output.

So this all works provided you put the endpoint in ~/.dockstore/config in my case no port:

[s3]
endpoint = https://XXX.sanger.ac.uk

and add include the tokens in the ~/.aws/credentials

[default]
aws_access_key_id = XXXXXX
aws_secret_access_key = XXXXXX

0reactions

denis-yuencommented, Apr 6, 2017

@garyluu I think the remaining task to finish this one off it to get your s3cmd plugin deployed on artifacts.oicr.on.ca

Top Results From Across the Web

Use ~/.s3cfg for non AWS S3 · Issue #548 · dockstore ... - GitHub

The documentation only shows AWS style configuration for S3 upload/download and doesn't appear to accept other methods. Desired behaviour. It ...

Configuring an S3 Client - Riak Docs

s3cfg file uses the Amazon S3 service as the storage backend. For a Riak CS system, change the following settings to point to...

About the s3cmd configuration file

The s3cmd configuration file is named .s3cfg and it is located in the user's home directory, e.g. /home/username/ ($HOME). On Windows the configuration...

Use AWS CLI and S3cmd with Vast

Now that we've setup our environment, let's go ahead and try some simple operations. List the root directory. s3cmd -c vast.s3cfg ls s3://...

using s3cmd with a different, s3 like, provider - Stack Overflow

Anyway recently I had to setup s3cmd on another machine and, not remembering where to set the endpoint, I grep the address of...