How to fix "Connection pool is full, discarding connection" warning?

Describe the bug

When I make more than 15 requests on locust in parallel, I get the warning

WARNING/urllib3.connectionpool: Connection pool is full, discarding connection:

which, according to a comment HERE, is caused by “logging”(?), and could be the reason of some underlying issue.

I tried to use the line logging.basicConfig(level=logging.ERROR) in the locustfile.py, but this does not solve the issue.

Expected behavior

The expected behavior is that locust is able to perform, like, 50 requests in parallel for 50 users. That is what I would expect from a performance testing tool.

Actual behavior

I get the warning

WARNING/urllib3.connectionpool: Connection pool is full, discarding connection:

and I am not sure locust is doing the correct thing (i.e. making like 20 requests in parallel).

Steps to reproduce

Here is the part of the code that is (hopefully) doing a bunch of requests in parallel (just like a browser would do is a user opens a web page):

    def make_parallel_requests(self, requests):
        group = Group()
        uid = str(uuid.uuid4())[:8]
        print(f"{uid} Starting {len(requests)} requests in parallel.")
        for request in requests:
            if request["method"] == "GET":
                url = request["url"]
                headers = request["headers"]
                headers["Authorization"] = "bearer " + ACCESS_TOKEN
                group.spawn(lambda: self.client.get(url, headers=headers, verify=False))
        group.join()
        print(f"{uid} Requests done")

It looks like that if the number of requests is smaller than 15 there is no such warning. But there will be printed that warning for every additional request.

Maybe I can ignore the warning and locust is, in fact, performing the requests in parallel?

Environment

• OS: MacOS 10.14.6
• Python version: 3.8.6
• Locust version: 1.4.1
• Locust command line that you ran: locust --headless -u 1 -r 1 -t 100s --csv-full-history --csv test --only-summary
• Locust file contents (anonymized if necessary): see above.

[aek]

I think that it will be better to allow users to provide an instance of their own HTTPAdapter instance. That way you could do any customization on your side like I have done in the __init__ method. I strongly believe that the FastHttpUser class could be easily converted into a requests HTTPAdapter and reuse all the stuffs already done in that matter. Using it like this you could use your own batteries to power on the same engine

[aek]

Hi @alex4200 Try it like this example(copy the __init__ method to your Locust User)

from requests.adapters import HTTPAdapter

class CustomUser(HttpUser):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.client.mount('https://', HTTPAdapter(pool_maxsize=50))
        self.client.mount('http://', HTTPAdapter(pool_maxsize=50))

    def make_parallel_requests(self, requests):
        group = Group()
        uid = str(uuid.uuid4())[:8]
        print(f"{uid} Starting {len(requests)} requests in parallel.")
        for request in requests:
            if request["method"] == "GET":
                url = request["url"]
                headers = request["headers"]
                headers["Authorization"] = "bearer " + ACCESS_TOKEN
                group.spawn(lambda: self.client.get(url, headers=headers, verify=False))
        group.join()
        print(f"{uid} Requests done")