Memory leak
See original GitHub issuePlease fill out the sections below to help us address your issue.
What issue did you see ? When I call the AWS interface in large numbers, and cannot connect to the corresponding area due to network reasons, an EndpointConnectionError is thrown. As time goes by, the memory occupied by my process will continue to increase, and the maximum observed is currently 6G. Using gc and pyrasite to check, it is found that gc.garbage is [], and the data type that takes up the most memory is str or unicode. The unicode description is the document description of DescribeInstancesRequest in service-2.json. The library versions I use: boto3-1.12.24, botocore-1.15.24, urllib3-1.21.1
Steps to reproduce If you have a runnable example, please include it as a snippet or link to a repository/gist for larger code examples.
Debug logs Full stack trace by adding
import botocore.session
botocore.session.Session().set_debug_logger('')
to your code.
Issue Analytics
- State:
- Created 3 years ago
- Comments:12 (3 by maintainers)
Is there an update or workaround for this problem?
I am also tracking down a memory leak.
tracemalloc
pointed me to https://github.com/boto/botocore/blob/develop/botocore/client.py#L322When running a Flask application and looping through
gc.garbage
after agc.collect()
I am left with boto docs. I currently am using boto3 and creating clients asclient = boto3.client('sts')
as an example.Running in lambda the following is a graph of memory from cloudwatch metrics filter: