Unhandled exception: stream ended at an unexpected time
See original GitHub issueDescribe the bug
I running a test pulling large files (some > 250mb) using the fast client, when connection gets dropped (“stream ended at an unexpected time”) on some files it throws an unhandled exception and does not log a failure.
Exception: Traceback (most recent call last): File “/home/mkarlovich/.pyenv/versions/3.8.2/lib/python3.8/site-packages/geventhttpclient/useragent.py”, line 184, in content return self._cached_content AttributeError: _cached_content
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File “/home/mkarlovich/.pyenv/versions/3.8.2/lib/python3.8/site-packages/locust/user/task.py”, line 284, in run self.execute_next_task() File “/home/mkarlovich/.pyenv/versions/3.8.2/lib/python3.8/site-packages/locust/user/task.py”, line 309, in execute_next_task self.execute_task(self._task_queue.pop(0)) File “/home/mkarlovich/.pyenv/versions/3.8.2/lib/python3.8/site-packages/locust/user/task.py”, line 416, in execute_task task(self.user) File “/home/mkarlovich/source/locust-plugins/examples/fast_csvreader_ex.py”, line 17, in index self.client.get(url, headers=headers) File “/home/mkarlovich/.pyenv/versions/3.8.2/lib/python3.8/site-packages/locust/contrib/fasthttp.py”, line 272, in get return self.request(“GET”, path, **kwargs) File “/home/mkarlovich/.pyenv/versions/3.8.2/lib/python3.8/site-packages/locust/contrib/fasthttp.py”, line 237, in request request_meta[“content_size”] = len(response.content or “”) File “/home/mkarlovich/.pyenv/versions/3.8.2/lib/python3.8/site-packages/geventhttpclient/useragent.py”, line 186, in content self._cached_content = self._content() File “/home/mkarlovich/.pyenv/versions/3.8.2/lib/python3.8/site-packages/locust/contrib/fasthttp.py”, line 340, in _content return super(FastResponse, self)._content() File “/home/mkarlovich/.pyenv/versions/3.8.2/lib/python3.8/site-packages/geventhttpclient/useragent.py”, line 201, in _content ret = self._response.read() File “/home/mkarlovich/.pyenv/versions/3.8.2/lib/python3.8/site-packages/geventhttpclient/response.py”, line 260, in read self.feed(data) _parser.HTTPParseError: (‘stream ended at an unexpected time’, 11)
Expected behavior
It should handle this exception and report it as a failure.
Actual behavior
Workers throw exception but no failure is recorded
Steps to reproduce
I can’t reproduce it except under load, code is using script below which constructs file download urls from csv file.
Environment
- OS: Ubuntu 16.04 master and workers
- Python version: 3.8.2
- Locust version: 1.0.3
- Locust command line that you ran:
locust -f fast_csvreader_ex.py --master --expect-workers 10 --headless -u 100 -r 1 -t 60m --csv=test100u60m10w locust -f fast_csvreader_ex.py --worker --master-host=10.66.11.21 & # run 5 per worker machine
- Locust file contents (anonymized if necessary):
from locust_plugins.csvreader import CSVReader
from locust import task
from locust.contrib.fasthttp import FastHttpUser
import traceback, logging, sys
import locust.stats
locust.stats.CSV_STATS_INTERVAL_SEC = 5
id_reader = CSVReader("ids.csv")
class MyUser(FastHttpUser):
@task
def index(self):
headers = { "Authorization": "apikey", "Accept": "application/json" }
feed_id, file_id = next(id_reader)
url = f'/feeds/{feed_id}/files/{file_id}'
self.client.get(url, headers=headers)
host = 'https://my-test-sever.com'
connection_timeout = 6000.0
network_timeout = 6000.0
Issue Analytics
- State:
- Created 3 years ago
- Comments:18 (9 by maintainers)
Top GitHub Comments
That works for me and marks wrong packets as a failure https://github.com/locustio/locust/pull/1464
@cyberw thank you! This is working for me as well.