question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Boto3 logs printing SecretString when log level is set to debug

See original GitHub issue

I’ve written a code which gets a secret from Secrets Manager. here’s the code snippet


import logging
import boto3

logging.basicConfig(format='%(asctime)s,%(msecs)d %(levelname)-8s'
                           ' [%(filename)s:%(lineno)d] %(message)s',
                    datefmt='%Y-%m-%d:%H:%M:%S',
                    level=logging.DEBUG)

logger = logging.getLogger()
logger.setLevel(logging.DEBUG)

database_creds = boto3 \
    .client(service_name="secretsmanager",
            region_name='us-west-2',
            aws_access_key_id='my-access-key',
            aws_secret_access_key='my-secret-key',
            aws_session_token='my-session-token') \
    .get_secret_value(SecretId='my-key') \
    .get('SecretString')

Now this code works fine but the problem I have is with logging. This produces log as below

2020-02-14:13:17:30,139 DEBUG    [auth.py:367] StringToSign:
AWS4-HMAC-SHA256
20200214T074730Z
20200214/us-west-2/secretsmanager/aws4_request
27fb77a9b42454155485fd050fcf5c20e2eb530ab767d9c82acc4bd61319d325
2020-02-14:13:17:30,140 DEBUG    [auth.py:369] Signature:
21c66d87431be49d3bea56e61824beb460fd49f86730e2fb67cb25d81098d120
2020-02-14:13:17:30,140 DEBUG    [endpoint.py:187] Sending http request: <AWSPreparedRequest stream_output=False, method=POST, url=https://secretsmanager.us-west-2.amazonaws.com/, headers={'X-Amz-Target': b'secretsmanager.GetSecretValue', 'Content-Type': b'application/x-amz-json-1.1', 'User-Agent': b'Boto3/1.11.5 Python/3.7.4 Windows/10 Botocore/1.14.5', 'X-Amz-Date': b'20200214T074730Z', 'X-Amz-Security-Token': b'FwoGZXIvYXdzEMn//////////wEaDLkioZ+JZ5nq', 'Authorization': b'AWS4-HMAC-SHA256 Credential=ASIAW7AJJER2YMO35N5M/20200214/us-west-2/secretsmanager/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-security-token;x-amz-target, Signature=21c66d87431be49d3bea56e61824beb460fd49f86730e2fb67cb25d81098d120', 'Content-Length': '47'}>
2020-02-14:13:17:30,141 DEBUG    [connectionpool.py:959] Starting new HTTPS connection (1): secretsmanager.us-west-2.amazonaws.com:443
2020-02-14:13:17:31,720 DEBUG    [connectionpool.py:437] https://secretsmanager.us-west-2.amazonaws.com:443 "POST / HTTP/1.1" 200 282
2020-02-14:13:17:31,721 DEBUG    [parsers.py:234] Response headers: {'Date': 'Fri, 14 Feb 2020 07:47:31 GMT', 'Content-Type': 'application/x-amz-json-1.1', 'Content-Length': '282', 'Connection': 'keep-alive', 'x-amzn-RequestId': '7f3cfcdd-7e63-4b80-83b4-48e7827b5ee6'}
2020-02-14:13:17:31,721 DEBUG    [parsers.py:235] Response body:
b'{"ARN":"arn:aws:secretsmanager:us-west-2:478908327029:secret:my-secret","CreatedDate":1.57531457696E9,"Name":"my-secret","SecretString":"YOU-SHOULDNT-BE-SEEING-THIS","VersionId":"9c5effcf-2d48-439d-8dae-2b0da7dbfb7f","VersionStages":["AWSCURRENT"]}'
2020-02-14:13:17:31,721 DEBUG    [hooks.py:210] Event needs-retry.secrets-manager.GetSecretValue: calling handler <botocore.retryhandler.RetryHandler object at 0x045FFCD0>
2020-02-14:13:17:31,721 DEBUG    [retryhandler.py:187] No retry needed.

Interesting line is this

2020-02-14:13:17:31,721 DEBUG    [parsers.py:235] Response body:
b'{"ARN":"arn:aws:secretsmanager:us-west-2:478908327029:secret:my-secret","CreatedDate":1.57531457696E9,"Name":"my-secret","SecretString":"YOU-SHOULDNT-BE-SEEING-THIS","VersionId":"9c5effcf-2d48-439d-8dae-2b0da7dbfb7f","VersionStages":["AWSCURRENT"]}'

Here, Secret is getting printed.

Irrespective of whether the debug mode is on or off. The secrets are never supposed to be get printed.

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Reactions:5
  • Comments:13 (4 by maintainers)

github_iconTop GitHub Comments

3reactions
rakeshbbcommented, Oct 22, 2022

Can we open this issue again, it’s not fixed even in October 2022.

3reactions
kujirajuunigoucommented, Oct 14, 2020

Any plans to address this yet? This looks like some pretty insecure defaults.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Lambda with boto3 logging secrets : r/aws - Reddit
I have a python lambda which uses boto3 to query secrets manager. ... The code uses "import logging" and sets the log level...
Read more >
Boto3 reference — Boto3 Docs 1.26.32 documentation - AWS
Add a stream handler for the given name and level to the logging module. ... For debugging purposes a good choice is to...
Read more >
How to get debug logs from boto3 in a local script?
This seems to work for me: import boto3 import logging boto3.set_stream_logger('', logging.DEBUG) s3_client = boto3.client('s3') for bucket ...
Read more >
Step 3 - Add logging to our code - AWS Workshop Studio
Set the default log level to debug and the log file to translate.log . Amend the if-elif-else statements to use logging with the...
Read more >
Reduce CloudWatch logs written by Glue job - AWS
Resolution · Choose the standard filter setting for continuous logging · Set the logging level using Spark context method setLogLevel · Use a ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found