question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

s3_iter_bucket for non default profile_name

See original GitHub issue

Hi,

I am trying to use s3_iter_bucket with the non default profile name.

Currently when I try to do it, I get a 403 Error.


import boto3
boto3.setup_default_session(profile_name='non-default-profile', region_name='eu-west-1')


from smart_open import s3_iter_bucket, open

bucket = 'some_bucket'
prefix = 'some_prefix'
key = 'some_prefix' + 'some_key'


session = boto3.DEFAULT_SESSION
client = session.client('s3')

# this works
some_object = client.get_object(Bucket=bucket, Key=key)

# this works
same_object = open('s3://' +  bucket + '/' + key, transport_params=dict(session=session))

# get access denied when I try do iterate over this
s3bucket_objs = s3_iter_bucket(bucket, prefix=prefix)

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:5 (1 by maintainers)

github_iconTop GitHub Comments

1reaction
piskvorkycommented, Oct 25, 2019

…Plus your smart_open version.

@mpenkov how about adding an issue template, similar to the one we have in gensim?

0reactions
mpenkovcommented, Mar 15, 2020

We will release a new version soon that resolves the problem. For now, you can do this with the current master:

smart_open.s3_iter_bucket(bucket_name, profile_name='non-default-profile', region_name='eu-west-1')
Read more comments on GitHub >

github_iconTop Results From Across the Web

Named profiles for the AWS CLI - AWS Command Line Interface
The first [default] is used when you run a AWS CLI command with no profile ... To use a named profile, add the...
Read more >
Using non-default profile · Issue #21 · boto/boto3 - GitHub
Is there a simple way to use a non-default profile by specifying it's ... import botocore.session import boto3 # Set your profile name...
Read more >
How do I set the name of the default profile in AWS CLI?
I have no DEFAULT profile specified in my ~/.aws/config , this was ... 'default' is the profile name given to your first profile...
Read more >
Configure Named AWS Profile & Usage in Applications
aws configure list [--profile profile-name]. If the — profile profile-name is not provided, it get the default profile $ aws configure list
Read more >
Switch to another Outlook email profile - Microsoft Support
... choose between multiple profiles or configure one profile as the default. ... the Profile Name list, click the profile that you want...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found