read_csv from S3 times out

data = dask.dataframe.read_csv('s3://test/test.csv')

times out with

ConnectTimeout: HTTPSConnectionPool(host='sts.amazonaws.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<botocore.awsrequest.AWSHTTPSConnection object at 0x7f999ca2c790>, 'Connection to sts.amazonaws.com timed out. (connect timeout=60)'))

however, the following works:

import s3fs
fs = s3fs.S3FileSystem()
fs.ls('test')

I am on ec2 instance which has an IAM role attached with S3FullAccess

[martindurant]

Seems to be this: botocore.utils.InstanceMetadataFetcher().retrieve_iam_role_credentials() https://github.com/boto/botocore/blob/develop/botocore/utils.py#L144

We still don’t have a way to know if we should distribute (temporary) credentials or not. I see that you suggest the user should have set their environment/config files/IAM roles correctly before starting. That’s what we initially had done too. But if the user provides the key/secret as overrides, sending these to workers might be fine or might be dangerous.

[hussainsultan]

@martindurant it was user error, i wasnt pulling it from conda-forge channel correctly. it works now. Thank you!