Loads of DEBUG messages with new s3transfer package
See original GitHub issueI just updated my boto3 and it installed a new s3transfer package (0.1.1). Problem is that now I get a load of DEBUG messages for every S3 download my app performs.
DEBUG:s3transfer.utils: Acquiring 0
DEBUG:s3transfer.tasks: DownloadSubmissionTask({'transfer_future': <s3transfer.futures.TransferFuture object at 0x10ae6a350>}) about to wait for the following futures []
DEBUG:s3transfer.tasks: DownloadSubmissionTask({'transfer_future': <s3transfer.futures.TransferFuture object at 0x10ae6a350>}) done waiting for dependent futures
DEBUG:s3transfer.tasks: Executing task DownloadSubmissionTask({'transfer_future': <s3transfer.futures.TransferFuture object at 0x10ae6a350>}) with kwargs {'io_executor': <s3transfer.futures.BoundedExecutor object at 0x10ae34f90>, 'request_executor': <s3transfer.futures.BoundedExecutor object at 0x10ae34b90>, 'osutil': <s3transfer.utils.OSUtils object at 0x10ae34950>, 'client': <botocore.client.S3 object at 0x10ae343d0>, 'transfer_future': <s3transfer.futures.TransferFuture object at 0x10ae6a350>, 'config': <boto3.s3.transfer.TransferConfig object at 0x10ae34910>}
DEBUG:s3transfer.futures: Submitting task GetObjectTask({'extra_args': {}, 'bucket': 'BUCKET', 'key': 'FILE'}) to executor <s3transfer.futures.BoundedExecutor object at 0x10ae34b90> for transfer request: 0.
DEBUG:s3transfer.utils: Acquiring 0
DEBUG:s3transfer.tasks: GetObjectTask({'extra_args': {}, 'bucket': 'BUCKET', 'key': 'FILE'}) about to wait for the following futures []
DEBUG:s3transfer.tasks: GetObjectTask({'extra_args': {}, 'bucket': 'BUCKET', 'key': 'FILE'}) done waiting for dependent futures
DEBUG:s3transfer.tasks: Executing task GetObjectTask({'extra_args': {}, 'bucket': 'BUCKET', 'key': 'FILE'}) with kwargs {'fileobj': <open file '/var/folders/kd/yx5n34z97kz10x25_5z9sq9m9qvmpm/T/tmpgrD02f.6A5DbD3C', mode 'wb' at 0x109865a50>, 'bucket': 'BUCKET', 'download_output_manager': <s3transfer.download.DownloadFilenameOutputManager object at 0x10ae90890>, 'extra_args': {}, 'callbacks': [], 'client': <botocore.client.S3 object at 0x10ae343d0>, 'key': 'FILE', 'io_chunksize': 262144, 'max_attempts': 5}
DEBUG:s3transfer.futures: Submitting task JoinFuturesTask({}) to executor <s3transfer.futures.BoundedExecutor object at 0x10ae34b90> for transfer request: 0.
DEBUG:s3transfer.utils: Acquiring 0
DEBUG:s3transfer.tasks: JoinFuturesTask({}) about to wait for the following futures [<Future at 0x10aeb9390 state=running>]
DEBUG:s3transfer.utils: Releasing acquire 0/None
DEBUG:s3transfer.tasks: JoinFuturesTask({}) about to wait for <Future at 0x10aeb9390 state=running>
DEBUG:s3transfer.futures: Submitting task IOWriteTask({'offset': 0}) to executor <s3transfer.futures.BoundedExecutor object at 0x10ae34f90> for transfer request: 0.
DEBUG:s3transfer.utils: Acquiring 0
DEBUG:s3transfer.tasks: IOWriteTask({'offset': 0}) about to wait for the following futures []
DEBUG:s3transfer.tasks: IOWriteTask({'offset': 0}) done waiting for dependent futures
DEBUG:s3transfer.tasks: Executing task IOWriteTask({'offset': 0}) with kwargs {'fileobj': <open file '/var/folders/kd/yx5n34z97kz10x25_5z9sq9m9qvmpm/T/tmpgrD02f.6A5DbD3C', mode 'wb' at 0x109865a50>, 'offset': 0}
DEBUG:s3transfer.utils: Releasing acquire 0/None
DEBUG:s3transfer.tasks: JoinFuturesTask({}) done waiting for dependent futures
DEBUG:s3transfer.utils: Releasing acquire 0/None
DEBUG:s3transfer.tasks: Executing task JoinFuturesTask({}) with kwargs {'futures_to_wait_on': [None]}
DEBUG:s3transfer.futures: Submitting task IORenameFileTask({'final_filename': '/var/folders/kd/yx5n34z97kz10x25_5z9sq9m9qvmpm/T/tmpgrD02f'}) to executor <s3transfer.futures.BoundedExecutor object at 0x10ae34f90> for transfer request: 0.
DEBUG:s3transfer.utils: Acquiring 0
DEBUG:s3transfer.utils: Releasing acquire 0/None
DEBUG:s3transfer.tasks: IORenameFileTask({'final_filename': '/var/folders/kd/yx5n34z97kz10x25_5z9sq9m9qvmpm/T/tmpgrD02f'}) about to wait for the following futures []
DEBUG:s3transfer.tasks: IORenameFileTask({'final_filename': '/var/folders/kd/yx5n34z97kz10x25_5z9sq9m9qvmpm/T/tmpgrD02f'}) done waiting for dependent futures
DEBUG:s3transfer.tasks: Executing task IORenameFileTask({'final_filename': '/var/folders/kd/yx5n34z97kz10x25_5z9sq9m9qvmpm/T/tmpgrD02f'}) with kwargs {'osutil': <s3transfer.utils.OSUtils object at 0x10ae34950>, 'final_filename': '/var/folders/kd/yx5n34z97kz10x25_5z9sq9m9qvmpm/T/tmpgrD02f', 'fileobj': <open file '/var/folders/kd/yx5n34z97kz10x25_5z9sq9m9qvmpm/T/tmpgrD02f.6A5DbD3C', mode 'wb' at 0x109865a50>}
DEBUG:s3transfer.utils: Releasing acquire 0/None
Issue Analytics
- State:
- Created 7 years ago
- Reactions:4
- Comments:10 (5 by maintainers)
Top Results From Across the Web
Disable boto logging without modifying the boto files
It is recommended to disable DEBUG and INFO messages from all external packages, not limited to botocore and boto3 : LOGGING_CONFIG = {...
Read more >Considerations in adopting RHEL 8 Red Hat Enterprise Linux 8
New syslog plug-in for logging any RPM activity by the System Logging protocol (syslog). The rpmbuild command can now do all build steps...
Read more >s3transfer - PyPI
S3transfer is a Python library for managing Amazon S3 transfers. This project is maintained and published by Amazon Web Services.
Read more >Using credentials - AWS SDK for Java 2.x
The SDK uses the SystemPropertyCredentialsProvider class to load credentials from the aws.accessKeyId and aws.secretAccessKey Java system properties.
Read more >8 Must-Know Tricks to Use S3 More Effectively in Python
Similarly, if you want to upload and read small pieces of textual data such as quotes, tweets, or news articles, you can do...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Set
s3transfer
toCRITICAL
only.Ahh I see what is happening. We need to add a null handler default in
s3transfer
. I sent a PR to fix this in thes3transfer
library.