session always uses AWS_PROFILE even if profile_name=None is set

Describe the bug I want to create a new credential set for my .aws/credentials file. Sometimes when I run the script to do so, I may already have environment variables set, pointing to the creds I am about to create. However I cannot prevent boto3 from trying to use the AWS_PROFILE. The error says the ${AWS_PROFILE} profile cannot be found. But I don’t want it to use that, that’s why I specified profile_name=None.

Steps to reproduce

export AWS_PROFILE=nonexistent
python3
>>> import boto3
>>> session = boto3.session.Session(profile_name=None,)

Expected behavior I’d expect an “empty” session to be created without a profile. Then I can do

>>> client = session.client('sts')
>>> response = client.assume_role_with_saml(
            RoleArn=role_arn,
....etc...
>>> session = boto3.Session(
    aws_access_key_id=response['Credentials']['AccessKeyId'],

Debug logs

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/site-packages/boto3/session.py", line 80, in __init__
    self._setup_loader()
  File "/usr/local/lib/python3.8/site-packages/boto3/session.py", line 120, in _setup_loader
    self._loader = self._session.get_component('data_loader')
  File "/usr/local/lib/python3.8/site-packages/botocore/session.py", line 682, in get_component
    return self._components.get_component(name)
  File "/usr/local/lib/python3.8/site-packages/botocore/session.py", line 919, in get_component
    self._components[name] = factory()
  File "/usr/local/lib/python3.8/site-packages/botocore/session.py", line 155, in <lambda>
    lambda:  create_loader(self.get_config_variable('data_path')))
  File "/usr/local/lib/python3.8/site-packages/botocore/session.py", line 237, in get_config_variable
    return self.get_component('config_store').get_config_variable(
  File "/usr/local/lib/python3.8/site-packages/botocore/configprovider.py", line 293, in get_config_variable
    return provider.provide()
  File "/usr/local/lib/python3.8/site-packages/botocore/configprovider.py", line 390, in provide
    value = provider.provide()
  File "/usr/local/lib/python3.8/site-packages/botocore/configprovider.py", line 451, in provide
    scoped_config = self._session.get_scoped_config()
  File "/usr/local/lib/python3.8/site-packages/botocore/session.py", line 337, in get_scoped_config
    raise ProfileNotFound(profile=profile_name)
botocore.exceptions.ProfileNotFound: The config profile (nonexistent) could not be found

[max-allan-surevine]

@swetashre the user has set the AWS_PROFILE already.

IF I don’t set any profile_name in the session then it uses the AWS_PROFILE and fails. If I set profile_name to None then it uses the AWS_PROFILE and fails.

MaAl00350:~ max $ export AWS_PROFILE=fud
MaAl00350:~ max $ 
MaAl00350:~ max $ python3
Python 3.8.5 (v3.8.5:580fbb018f, Jul 20 2020, 12:11:27) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import boto3
>>> session = boto3.session.Session()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/boto3/session.py", line 80, in __init__
    self._setup_loader()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/boto3/session.py", line 120, in _setup_loader
    self._loader = self._session.get_component('data_loader')
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/botocore/session.py", line 682, in get_component
    return self._components.get_component(name)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/botocore/session.py", line 919, in get_component
    self._components[name] = factory()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/botocore/session.py", line 155, in <lambda>
    lambda:  create_loader(self.get_config_variable('data_path')))
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/botocore/session.py", line 237, in get_config_variable
    return self.get_component('config_store').get_config_variable(
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/botocore/configprovider.py", line 293, in get_config_variable
    return provider.provide()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/botocore/configprovider.py", line 390, in provide
    value = provider.provide()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/botocore/configprovider.py", line 451, in provide
    scoped_config = self._session.get_scoped_config()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/botocore/session.py", line 337, in get_scoped_config
    raise ProfileNotFound(profile=profile_name)
botocore.exceptions.ProfileNotFound: The config profile (fud) could not be found

[swetashre]

Sorry if my comments were not clear. I meant if you don’t specify any profile name then the default profile will be used. But if there is no default profile, an empty config dictionary will be used to create a session. So yes, you can create an empty session. If you want to create an empty session then don’t set the profile_name parameter at all.

https://boto3.amazonaws.com/v1/documentation/api/latest/guide/session.html

Note

Only set the profile_name parameter when a specific profile is required for your session. To use the default profile, don’t set the profile_name parameter at all. If the profile_name parameter isn't set and there is no default profile, an empty config dictionary will be used.

For your second point:

If I delete the AWS_PROFILE, I can call assume_role_with_saml without any credentials. (At the point that I assume the role I have no credentials. It makes no sense to require credentials to gain credentials).
It is only if I specify an AWS_PROFILE that doesn't exist that it fails. How can I make it use whichever profile it uses when AWS_PROFILE is not set.

I agree assume_role_with_saml api does not need any credentials but api calls like sts assume_role api need credentials to make initial api call. So in this case

client.assume_role(RoleArn='arn:aws:iam::123456789012:role/test_assume_role',RoleSessionName='test')

i will get error as i created an empty session without any credentials.

So in your case if you don’t want the session to use any AWS_PROFILE then don’t set the profile_name parameter at all.

I hope it helps and please let me know if you have any concerns.