Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Error during fetching a job

See original GitHub issue

My actions before raising this issue

  • Read/searched the docs
  • Searched past issues

image

Encounter this error while multiple users open different jobs in a task at nearly the same time while cooperating. And I have tested and got the same error in my machine while open multiple jobs. Everything is ok while open this jobs one by one.

Expected Behaviour

Open multiple jobs in a task at nearly the same time.

Current Behaviour

Jobs open failed

Possible Solution

Steps to Reproduce (for bugs)

  1. Create a task use connected file share without copy data into CVAT ( 140k images, 1000 segment size , partial annotated in my case)
  2. upload CVAT 1.1 annotation.zip
  3. Open multiple jobs at the same time
  4. The error occurs then

Context

Your Environment

  • Git hash commit (git log -1): e78bd06
  • Docker version docker version (e.g. Docker 17.0.05): 20.10.6
  • Are you using Docker Swarm or Kubernetes? no
  • Operating System and version (e.g. Linux, Windows, MacOS): Linux
  • Code example or link to GitHub repo or gist to reproduce problem:
  • Other diagnostic information / logs:
    Logs from `cvat` container
Here is the `docker logs cvat`

2021-08-26 01:29:59,823 DEBG ‘rqworker_low’ stderr output: DEBUG - 2021-08-26 01:29:59,822 - worker - Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.

2021-08-26 01:30:39,402 DEBG ‘rqworker_default_0’ stderr output: DEBUG - 2021-08-26 01:30:39,401 - worker - Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.

2021-08-26 01:35:54,264 DEBG ‘rqworker_default_1’ stderr output: DEBUG - 2021-08-26 01:35:54,263 - worker - Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds. 2021-08-26 01:36:20,652 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:36:20.652200 2021] [wsgi:error] [pid 668:tid 140472099108608] [remote 172.28.0.6:46604] INFO - 2021-08-26 01:36:20,652 - views - {“client_id”:731816,“name”:“Send user activity”,“time”:“2021-08-26T01:36:20.655000Z”,“payload”:{“working_time”:0},“is_active”:true,“username”:“user2”}

2021-08-26 01:36:24,096 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:36:24.096183 2021] [wsgi:error] [pid 668:tid 140472132679424] [remote 172.28.0.6:46626] [2021-08-26 01:36:24,096] INFO cvat.server.task_247: get repository request

2021-08-26 01:36:24,096 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:36:24.096302 2021] [wsgi:error] [pid 668:tid 140472132679424] [remote 172.28.0.6:46626] INFO - 2021-08-26 01:36:24,096 - views - get repository request

2021-08-26 01:36:34,985 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:36:34.985331 2021] [wsgi:error] [pid 668:tid 140472132679424] [remote 172.28.0.6:47470] INFO - 2021-08-26 01:36:34,985 - views - {“client_id”:794438,“name”:“Send user activity”,“time”:“2021-08-26T01:36:34.960000Z”,“payload”:{“working_time”:0},“is_active”:true,“username”:“user2”}

2021-08-26 01:36:35,913 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:36:35.913275 2021] [wsgi:error] [pid 668:tid 140472124286720] [remote 172.28.0.6:47482] [2021-08-26 01:36:35,907] INFO cvat.server.task_247: get repository request

2021-08-26 01:36:35,927 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:36:35.927765 2021] [wsgi:error] [pid 668:tid 140472124286720] [remote 172.28.0.6:47482] INFO - 2021-08-26 01:36:35,907 - views - get repository request

2021-08-26 01:36:38,808 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:36:38.808329 2021] [wsgi:error] [pid 668:tid 140472132679424] [remote 172.28.0.6:47510] INFO - 2021-08-26 01:36:38,808 - views - {“client_id”:842660,“name”:“Send user activity”,“time”:“2021-08-26T01:34:48.628000Z”,“payload”:{“working_time”:11234},“is_active”:true,“username”:“user1”}

2021-08-26 01:36:44,870 DEBG ‘rqworker_low’ stderr output: DEBUG - 2021-08-26 01:36:44,869 - worker - Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.

2021-08-26 01:37:24,555 DEBG ‘rqworker_default_0’ stderr output: DEBUG - 2021-08-26 01:37:24,555 - worker - Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.

2021-08-26 01:37:35,234 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:37:35.233880 2021] [wsgi:error] [pid 591:tid 140472235116288] [client 172.28.0.6:47478] Timeout when reading response headers from daemon process ‘localhost:8080’: /tmp/cvat-server/htdocs/api, referer: http://192.168.4.77:8080/tasks/247/jobs/1584

2021-08-26 01:37:38,809 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:37:38.808955 2021] [wsgi:error] [pid 549:tid 140472235656960] [client 172.28.0.6:47512] Timeout when reading response headers from daemon process ‘localhost:8080’: /tmp/cvat-server/htdocs/api, referer: http://192.168.4.77:8080/tasks/247/jobs/1523

2021-08-26 01:37:42,308 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:37:42.308720 2021] [wsgi:error] [pid 668:tid 140472107501312] [remote 172.28.0.6:47616] INFO - 2021-08-26 01:37:42,308 - views - {“client_id”:752849,“name”:“Send user activity”,“time”:“2021-08-26T01:35:53.251000Z”,“payload”:{“working_time”:0},“is_active”:true,“username”:“user1”}

2021-08-26 01:37:47,425 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:37:47.424882 2021] [wsgi:error] [pid 668:tid 140472132679424] [remote 172.28.0.6:47658] INFO - 2021-08-26 01:37:47,424 - views - {“client_id”:863581,“name”:“Send user activity”,“time”:“2021-08-26T01:37:47.419000Z”,“payload”:{“working_time”:0},“is_active”:true,“username”:“user2”}

2021-08-26 01:38:26,649 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:38:26.649349 2021] [wsgi:info] [pid 668:tid 140472149464832] mod_wsgi (pid=668): Daemon process request time limit exceeded, stopping process ‘localhost:8080’.

2021-08-26 01:38:26,649 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:38:26.649439 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): Shutdown requested ‘localhost:8080’.

2021-08-26 01:38:26,649 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:38:26.649817 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): Dumping stack trace for active Python threads. [Thu Aug 26 01:38:26.649830 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): Thread 140472124286720 executing file “/opt/venv/lib/python3.8/site-packages/asgiref/local.py”, line 102, in getattr

2021-08-26 01:38:26,650 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:38:26.649837 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/utils.py”, line 208, in getitem, [Thu Aug 26 01:38:26.649843 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/sql/query.py”, line 1177, in build_lookup, [Thu Aug 26 01:38:26.649848 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/sql/query.py”, line 1319, in build_filter, [Thu Aug 26 01:38:26.649854 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/sql/query.py”, line 1377, in _add_q, [Thu Aug 26 01:38:26.649859 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/sql/query.py”, line 1358, in add_q, [Thu Aug 26 01:38:26.649865 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 969, in _filter_or_exclude_inplace, [Thu Aug 26 01:38:26.649871 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 962, in _filter_or_exclude, [Thu Aug 26 01:38:26.649876 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 942, in filter, [Thu Aug 26 01:38:26.649882 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py”, line 584, in _apply_rel_filters, [Thu Aug 26 01:38:26.649888 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py”, line 619, in get_queryset, [Thu Aug 26 01:38:26.649893 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1890, in prefetch_one_level, [Thu Aug 26 01:38:26.649899 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1710, in prefetch_related_objects, [Thu Aug 26 01:38:26.649905 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 814, in _prefetch_related_objects, [Thu Aug 26 01:38:26.649910 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1310, in _fetch_all, [Thu Aug 26 01:38:26.649926 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/cacheops/query.py”, line 272, in _fetch_all, [Thu Aug 26 01:38:26.649932 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 287, in iter, [Thu Aug 26 01:38:26.649937 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py”, line 636, in get_prefetch_queryset, [Thu Aug 26 01:38:26.649943 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1823, in prefetch_one_level, [Thu Aug 26 01:38:26.649948 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1710, in prefetch_related_objects, [Thu Aug 26 01:38:26.649954 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 814, in _prefetch_related_objects, [Thu Aug 26 01:38:26.649962 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1310, in _fetch_all, [Thu Aug 26 01:38:26.649969 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/cacheops/query.py”, line 285, in _fetch_all, [Thu Aug 26 01:38:26.649977 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 269, in len, [Thu Aug 26 01:38:26.649983 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 425, in get, [Thu Aug 26 01:38:26.649989 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/cacheops/query.py”, line 352, in get, [Thu Aug 26 01:38:26.649995 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/home/django/cvat/apps/engine/views.py”, line 756, in data_info, [Thu Aug 26 01:38:26.650001 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/rest_framework/views.py”, line 502, in dispatch, [Thu Aug 26 01:38:26.650006 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/rest_framework/viewsets.py”, line 114, in view,

2021-08-26 01:38:26,650 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:38:26.650012 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/views/decorators/csrf.py”, line 54, in wrapped_view, [Thu Aug 26 01:38:26.650018 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/base.py”, line 181, in _get_response, [Thu Aug 26 01:38:26.650024 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650030 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650036 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650043 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650051 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650057 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650062 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650067 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650073 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650078 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650084 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650089 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650094 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650100 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650105 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650110 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650116 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650123 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650130 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650135 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/base.py”, line 130, in get_response, [Thu Aug 26 01:38:26.650141 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/wsgi.py”, line 133, in call, [Thu Aug 26 01:38:26.650147 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/mod_wsgi/server/init.py”, line 1514, in handle_request. [Thu Aug 26 01:38:26.650153 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): Thread 140472115894016 executing file “/opt/venv/lib/python3.8/site-packages/asgiref/local.py”, line 102, in getattr [Thu Aug 26 01:38:26.650161 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/utils.py”, line 208, in getitem, [Thu Aug 26 01:38:26.650168 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/sql/query.py”, line 2298, in is_nullable, [Thu Aug 26 01:38:26.650173 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/sql/query.py”, line 1604, in setup_joins, [Thu Aug 26 01:38:26.650179 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/sql/query.py”, line 1284, in build_filter, [Thu Aug 26 01:38:26.650184 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/sql/query.py”, line 1377, in _add_q,

2021-08-26 01:38:26,650 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:38:26.650189 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/sql/query.py”, line 1358, in add_q, [Thu Aug 26 01:38:26.650195 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 969, in _filter_or_exclude_inplace, [Thu Aug 26 01:38:26.650200 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 962, in _filter_or_exclude, [Thu Aug 26 01:38:26.650205 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 942, in filter, [Thu Aug 26 01:38:26.650211 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py”, line 584, in _apply_rel_filters, [Thu Aug 26 01:38:26.650216 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py”, line 619, in get_queryset, [Thu Aug 26 01:38:26.650221 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1890, in prefetch_one_level, [Thu Aug 26 01:38:26.650228 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1710, in prefetch_related_objects, [Thu Aug 26 01:38:26.650235 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 814, in _prefetch_related_objects, [Thu Aug 26 01:38:26.650242 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1310, in _fetch_all, [Thu Aug 26 01:38:26.650247 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/cacheops/query.py”, line 272, in _fetch_all, [Thu Aug 26 01:38:26.650253 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 287, in iter, [Thu Aug 26 01:38:26.650258 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py”, line 636, in get_prefetch_queryset, [Thu Aug 26 01:38:26.650264 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1823, in prefetch_one_level, [Thu Aug 26 01:38:26.650269 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1710, in prefetch_related_objects, [Thu Aug 26 01:38:26.650274 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 814, in _prefetch_related_objects, [Thu Aug 26 01:38:26.650280 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1310, in _fetch_all, [Thu Aug 26 01:38:26.650285 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/cacheops/query.py”, line 285, in _fetch_all, [Thu Aug 26 01:38:26.650290 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 269, in len, [Thu Aug 26 01:38:26.650296 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 425, in get, [Thu Aug 26 01:38:26.650302 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/cacheops/query.py”, line 352, in get, [Thu Aug 26 01:38:26.650307 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/home/django/cvat/apps/engine/views.py”, line 756, in data_info, [Thu Aug 26 01:38:26.650312 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/rest_framework/views.py”, line 502, in dispatch, [Thu Aug 26 01:38:26.650318 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/rest_framework/viewsets.py”, line 114, in view, [Thu Aug 26 01:38:26.650323 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/views/decorators/csrf.py”, line 54, in wrapped_view, [Thu Aug 26 01:38:26.650329 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/base.py”, line 181, in _get_response, [Thu Aug 26 01:38:26.650334 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650340 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650345 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650351 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650356 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650361 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call,

2021-08-26 01:38:26,650 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:38:26.650367 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650373 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650379 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650384 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650390 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650395 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650400 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650405 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650412 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650420 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650425 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650430 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650436 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650441 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/base.py”, line 130, in get_response, [Thu Aug 26 01:38:26.650446 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/wsgi.py”, line 133, in call, [Thu Aug 26 01:38:26.650452 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/mod_wsgi/server/init.py”, line 1514, in handle_request. [Thu Aug 26 01:38:26.650457 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): Thread 140472099108608 executing file “/opt/venv/lib/python3.8/site-packages/asgiref/local.py”, line 102, in getattr [Thu Aug 26 01:38:26.650462 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/utils.py”, line 208, in getitem, [Thu Aug 26 01:38:26.650468 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py”, line 580, in _apply_rel_filters, [Thu Aug 26 01:38:26.650473 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py”, line 619, in get_queryset, [Thu Aug 26 01:38:26.650479 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1890, in prefetch_one_level, [Thu Aug 26 01:38:26.650486 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1710, in prefetch_related_objects, [Thu Aug 26 01:38:26.650493 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 814, in _prefetch_related_objects, [Thu Aug 26 01:38:26.650498 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1310, in _fetch_all, [Thu Aug 26 01:38:26.650503 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/cacheops/query.py”, line 272, in _fetch_all, [Thu Aug 26 01:38:26.650509 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 287, in iter, [Thu Aug 26 01:38:26.650514 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py”, line 636, in get_prefetch_queryset, [Thu Aug 26 01:38:26.650520 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1823, in prefetch_one_level, [Thu Aug 26 01:38:26.650526 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1710, in prefetch_related_objects, [Thu Aug 26 01:38:26.650533 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 814, in _prefetch_related_objects, [Thu Aug 26 01:38:26.650538 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1310, in _fetch_all, [Thu Aug 26 01:38:26.650544 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/cacheops/query.py”, line 285, in _fetch_all, [Thu Aug 26 01:38:26.650549 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 269, in len, [Thu Aug 26 01:38:26.650555 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 425, in get, [3860/9799]

2021-08-26 01:38:26,650 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:38:26.650560 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/cacheops/query.py”, line 352, in get, [Thu Aug 26 01:38:26.650566 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/home/django/cvat/apps/engine/views.py”, line 756, in data_info, [Thu Aug 26 01:38:26.650571 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/rest_framework/views.py”, line 502, in dispatch, [Thu Aug 26 01:38:26.650577 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/rest_framework/viewsets.py”, line 114, in view, [Thu Aug 26 01:38:26.650582 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/views/decorators/csrf.py”, line 54, in wrapped_view, [Thu Aug 26 01:38:26.650588 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/base.py”, line 181, in _get_response, [Thu Aug 26 01:38:26.650593 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650599 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650606 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650612 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650617 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650622 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650628 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650633 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650639 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650644 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650649 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650655 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650660 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650665 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650670 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650678 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650684 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650690 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650695 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650700 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/base.py”, line 130, in get_response, [Thu Aug 26 01:38:26.650706 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/wsgi.py”, line 133, in call, [Thu Aug 26 01:38:26.650711 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/mod_wsgi/server/init.py”, line 1514, in handle_request. [Thu Aug 26 01:38:26.650717 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): Thread 140472132679424 executing file “/opt/venv/lib/python3.8/site-packages/asgiref/local.py”, line 102, in getattr [Thu Aug 26 01:38:26.650722 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/utils.py”, line 209, in getitem, [Thu Aug 26 01:38:26.650734 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py”, line 580, in _apply_rel_filters, [Thu Aug 26 01:38:26.650740 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py”, line 619, in get_queryset, [Thu Aug 26 01:38:26.650746 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1890, in prefetch_one_level,

2021-08-26 01:38:26,650 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:38:26.650753 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1710, in prefetch_related_objects, [Thu Aug 26 01:38:26.650759 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 814, in _prefetch_related_objects, [Thu Aug 26 01:38:26.650765 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1310, in _fetch_all, [Thu Aug 26 01:38:26.650770 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/cacheops/query.py”, line 272, in _fetch_all, [Thu Aug 26 01:38:26.650776 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 287, in iter, [Thu Aug 26 01:38:26.650781 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py”, line 636, in get_prefetch_queryset, [Thu Aug 26 01:38:26.650786 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1823, in prefetch_one_level, [Thu Aug 26 01:38:26.650792 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1710, in prefetch_related_objects, [Thu Aug 26 01:38:26.650797 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 814, in _prefetch_related_objects, [Thu Aug 26 01:38:26.650803 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 1310, in _fetch_all, [Thu Aug 26 01:38:26.650808 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/cacheops/query.py”, line 285, in _fetch_all, [Thu Aug 26 01:38:26.650813 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 269, in len, [Thu Aug 26 01:38:26.650819 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/db/models/query.py”, line 425, in get, [Thu Aug 26 01:38:26.650825 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/cacheops/query.py”, line 352, in get, [Thu Aug 26 01:38:26.650832 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/home/django/cvat/apps/engine/views.py”, line 756, in data_info, [Thu Aug 26 01:38:26.650837 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/rest_framework/views.py”, line 502, in dispatch, [Thu Aug 26 01:38:26.650843 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/rest_framework/viewsets.py”, line 114, in view, [Thu Aug 26 01:38:26.650848 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/views/decorators/csrf.py”, line 54, in wrapped_view, [Thu Aug 26 01:38:26.650854 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/base.py”, line 181, in _get_response, [Thu Aug 26 01:38:26.650859 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650865 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650870 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650876 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650881 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650886 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650892 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650897 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650905 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650910 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650916 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650921 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650926 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650932 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call,

2021-08-26 01:38:26,651 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:38:26.650937 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650943 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650948 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650953 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/utils/deprecation.py”, line 114, in call, [Thu Aug 26 01:38:26.650959 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/exception.py”, line 47, in inner, [Thu Aug 26 01:38:26.650964 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/base.py”, line 130, in get_response, [Thu Aug 26 01:38:26.650970 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/django/core/handlers/wsgi.py”, line 133, in call, [Thu Aug 26 01:38:26.650977 2021] [wsgi:info] [pid 668:tid 140472242674752] mod_wsgi (pid=668): called from file “/opt/venv/lib/python3.8/site-packages/mod_wsgi/server/init.py”, line 1514, in handle_request.

2021-08-26 01:38:31,649 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:38:31.649639 2021] [wsgi:info] [pid 668:tid 140469939369728] mod_wsgi (pid=668): Aborting process ‘localhost:8080’. [Thu Aug 26 01:38:31.649662 2021] [wsgi:info] [pid 668:tid 140469939369728] mod_wsgi (pid=668): Exiting process ‘localhost:8080’.

2021-08-26 01:38:33,182 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:38:33.182003 2021] [wsgi:error] [pid 549:tid 140472235116288] [client 172.28.0.6:47624] Truncated or oversized response headers received from daemon process ‘localhost:8080’: /tmp/cvat-server/htdocs/api, referer: http://192.168.4.77:8080/tasks/247/jobs/1523 [Thu Aug 26 01:38:33.182070 2021] [wsgi:error] [pid 591:tid 140472229549824] [client 172.28.0.6:47666] Truncated or oversized response headers received from daemon process ‘localhost:8080’: /tmp/cvat-server/htdocs/api, referer: http://192.168.4.77:8080/tasks/247/jobs/1584

2021-08-26 01:38:33,264 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:38:33.264105 2021] [wsgi:info] [pid 15:tid 140472242674752] mod_wsgi (pid=668): Process ‘localhost:8080’ has died, deregister and restart it. [Thu Aug 26 01:38:33.264146 2021] [wsgi:info] [pid 15:tid 140472242674752] mod_wsgi (pid=668): Process ‘localhost:8080’ has been deregistered and will no longer be monitored.

2021-08-26 01:38:33,280 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:38:33.264665 2021] [wsgi:info] [pid 757:tid 140472242674752] mod_wsgi (pid=757): Starting process ‘localhost:8080’ with threads=5.

2021-08-26 01:38:33,298 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:38:33.297921 2021] [wsgi:info] [pid 757:tid 140472242674752] mod_wsgi (pid=757): Python home /opt/venv.

2021-08-26 01:38:33,298 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:38:33.298034 2021] [wsgi:info] [pid 757:tid 140472242674752] mod_wsgi (pid=757): Initializing Python.

2021-08-26 01:38:33,367 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:38:33.367254 2021] [wsgi:info] [pid 757:tid 140472242674752] mod_wsgi (pid=757): Attach interpreter ‘’.

2021-08-26 01:38:33,395 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:38:33.395392 2021] [wsgi:info] [pid 757:tid 140472242674752] mod_wsgi (pid=757): Imported ‘mod_wsgi’.

2021-08-26 01:38:33,406 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:38:33.406011 2021] [wsgi:info] [pid 757:tid 140472242674752] mod_wsgi (pid=757, process=‘localhost:8080’, application=‘’): Loading Python script file ‘/tmp/cvat-server/handler.wsgi’.

2021-08-26 01:38:34,429 DEBG ‘ssh-agent’ stderr output: debug2: fd 4 setting O_NONBLOCK

2021-08-26 01:38:34,450 DEBG ‘ssh-agent’ stderr output: debug1: process_message: socket 1 (fd=4) type 17

2021-08-26 01:38:34,454 DEBG ‘ssh-agent’ stderr output: debug1: process_add_identity: add ssh-rsa SHA256:WJbmu830WDQAsHWcsguWuSCRyYOltWfbNjBzE2dp9w4 “django@b86a70feb6ac” (life: 0) (confirm: 0) (provider: none)

2021-08-26 01:38:34,485 DEBG ‘ssh-agent’ stderr output: debug2: fd 4 setting O_NONBLOCK debug1: process_message: socket 1 (fd=4) type 11

2021-08-26 01:38:53,452 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:38:53.452045 2021] [wsgi:error] [pid 757:tid 140472132679424] [remote 172.28.0.6:47670] INFO - 2021-08-26 01:38:53,451 - views - {“client_id”:731816,“name”:“Send user activity”,“time”:“2021-08-26T01:38:42.142000Z”,“payload”:{“working_time”:6846},“is_active”:true,“username”:“user2”}

2021-08-26 01:39:42,190 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:39:42.190246 2021] [wsgi:error] [pid 549:tid 140472235390720] [client 172.28.0.6:47672] Timeout when reading response headers from daemon process ‘localhost:8080’: /tmp/cvat-server/htdocs/api, referer: http://192.168.4.77:8080/tasks/247/jobs/1584

2021-08-26 01:39:58,304 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:39:58.304191 2021] [wsgi:error] [pid 757:tid 140472124286720] [remote 172.28.0.6:47750] [2021-08-26 01:39:58,304] INFO cvat.server.task_247: get repository request

2021-08-26 01:39:58,304 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:39:58.304302 2021] [wsgi:error] [pid 757:tid 140472124286720] [remote 172.28.0.6:47750] INFO - 2021-08-26 01:39:58,304 - views - get repository request

2021-08-26 01:40:07,635 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:40:07.634900 2021] [wsgi:error] [pid 757:tid 140472107501312] [remote 172.28.0.6:48562] INFO - 2021-08-26 01:40:07,634 - views - {“client_id”:994276,“name”:“Send user activity”,“time”:“2021-08-26T01:40:07.640000Z”,“payload”:{“working_time”:7659},“is_active”:true,“username”:“user2”}

2021-08-26 01:40:54,313 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:40:54.312985 2021] [wsgi:error] [pid 757:tid 140472115894016] [remote 172.28.0.6:48598] INFO - 2021-08-26 01:40:54,307 - views - {“job_id”:1637,“task_id”:247,“client_id”:179920,“name”:“Change frame”,“time”:“2021-08-26T01:34:56.025000Z”,“payload”:{“from”:120460,“to”:120490},“is_active”:true,“username”:" user3"}

2021-08-26 01:40:54,323 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:40:54.323403 2021] [wsgi:error] [pid 757:tid 140472115894016] [remote 172.28.0.6:48598] INFO - 2021-08-26 01:40:54,318 - views - {“job_id”:1637,“task_id”:247,“client_id”:179920,“name”:“Fit image”,“time”:“2021-08-26T01:34:56.170000Z”,“payload”:{},“is_active”:true,“username”:“user3”}

2021-08-26 01:40:54,333 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:40:54.333740 2021] [wsgi:error] [pid 757:tid 140472115894016] [remote 172.28.0.6:48598] INFO - 2021-08-26 01:40:54,328 - views - {“job_id”:1637,“task_id”:247,“client_id”:179920,“name”:“Change frame”,“time”:“2021-08-26T01:34:56.176000Z”,“payload”:{“from”:120490,“to”:120520},“is_active”:true,“username”:" user3"}

2021-08-26 01:40:54,344 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:40:54.344151 2021] [wsgi:error] [pid 757:tid 140472115894016] [remote 172.28.0.6:48598] INFO - 2021-08-26 01:40:54,338 - views - {“job_id”:1637,“task_id”:247,“client_id”:179920,“name”:“Fit image”,“time”:“2021-08-26T01:34:58.604000Z”,“payload”:{},“is_active”:true,“username”:“user3”}

2021-08-26 01:40:54,354 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:40:54.354467 2021] [wsgi:error] [pid 757:tid 140472115894016] [remote 172.28.0.6:48598] INFO - 2021-08-26 01:40:54,349 - views - {“job_id”:1637,“task_id”:247,“client_id”:179920,“name”:“Change frame”,“time”:“2021-08-26T01:34:59.494000Z”,“payload”:{“from”:120520,“to”:120550},“is_active”:true,“username”:" user3"}

2021-08-26 01:40:54,364 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:40:54.364862 2021] [wsgi:error] [pid 757:tid 140472115894016] [remote 172.28.0.6:48598] INFO - 2021-08-26 01:40:54,359 - views - {“job_id”:1637,“task_id”:247,“client_id”:179920,“name”:“Fit image”,“time”:“2021-08-26T01:35:02.057000Z”,“payload”:{},“is_active”:true,“username”:“user3”}

2021-08-26 01:40:54,375 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:40:54.375234 2021] [wsgi:error] [pid 757:tid 140472115894016] [remote 172.28.0.6:48598] INFO - 2021-08-26 01:40:54,370 - views - {“job_id”:1637,“task_id”:247,“client_id”:179920,“name”:“Change frame”,“time”:“2021-08-26T01:35:02.541000Z”,“payload”:{“from”:120550,“to”:120580},“is_active”:true,“username”:" user3"}

2021-08-26 01:40:54,385 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:40:54.385651 2021] [wsgi:error] [pid 757:tid 140472115894016] [remote 172.28.0.6:48598] INFO - 2021-08-26 01:40:54,380 - views - {“job_id”:1637,“task_id”:247,“client_id”:179920,“name”:“Fit image”,“time”:“2021-08-26T01:35:05.345000Z”,“payload”:{},“is_active”:true,“username”:“user3”}

2021-08-26 01:40:54,396 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:40:54.396171 2021] [wsgi:error] [pid 757:tid 140472115894016] [remote 172.28.0.6:48598] INFO - 2021-08-26 01:40:54,390 - views - {“job_id”:1637,“task_id”:247,“client_id”:179920,“name”:“Change frame”,“time”:“2021-08-26T01:35:06.172000Z”,“payload”:{“from”:120580,“to”:120610},“is_active”:true,“username”:" user3"}

2021-08-26 01:40:54,406 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:40:54.406588 2021] [wsgi:error] [pid 757:tid 140472115894016] [remote 172.28.0.6:48598] INFO - 2021-08-26 01:40:54,401 - views - {“job_id”:1637,“task_id”:247,“client_id”:179920,“name”:“Fit image”,“time”:“2021-08-26T01:35:07.748000Z”,“payload”:{},“is_active”:true,“username”:“user3”}

2021-08-26 01:40:54,412 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:40:54.411957 2021] [wsgi:error] [pid 757:tid 140472115894016] [remote 172.28.0.6:48598] INFO - 2021-08-26 01:40:54,411 - views - {“job_id”:1637,“task_id”:247,“client_id”:179920,“name”:“Change frame”,“time”:“2021-08-26T01:35:09.927000Z”,“payload”:{“from”:120610,“to”:120611},“is_active”:true,“username”:" user3"}

2021-08-26 01:40:54,422 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:40:54.422354 2021] [wsgi:error] [pid 757:tid 140472115894016] [remote 172.28.0.6:48598] INFO - 2021-08-26 01:40:54,417 - views - {“job_id”:1637,“task_id”:247,“client_id”:179920,“name”:“Fit image”,“time”:“2021-08-26T01:35:09.983000Z”,“payload”:{},“is_active”:true,“username”:“user3”}

2021-08-26 01:40:54,432 DEBG ‘runserver’ stderr output: [Thu Aug 26 01:40:54.432759 2021] [wsgi:error] [pid 757:tid 140472115894016] [remote 172.28.0.6:48598] INFO - 2021-08-26 01:40:54,427 - views - {“job_id”:1637,“task_id”:247,“client_id”:179920,“name”:“Change frame”,“time”:“2021-08-26T01:35:10.133000Z”,“payload”:{“from”:120611,“to”:120612},“is_active”:true,“username”:" user3"}

Next steps

You may join our Gitter channel for community support.

Issue Analytics

  • State:open
  • Created 2 years ago
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

2reactions
PhamThinh31commented, Dec 1, 2021

Hi everyone, I have tested and edited the supervisord.conf file to increase the number of processes and threads --processes 3 --threads 5. This works for me.

0reactions
sa4zetcommented, Jun 28, 2022

You can set any extra command line argument in DJANGO_MODWSGI_EXTRA_ARGS environment variable. like DJANGO_MODWSGI_EXTRA_ARGS=--processes 4, but this didn’t solve the problem…

Read more comments on GitHub >

github_iconTop Results From Across the Web

Error during fetching a job · Issue #3604 · opencv/cvat - GitHub
Encounter this error while multiple users open different jobs in a task at nearly the same time while cooperating.
Read more >
Backend: An error occurred while fetching the job and viewing ...
For a job that shows "An error occurred while fetching the job" when viewing it and has the 500 when attempting to view...
Read more >
"Error while fetching jobs from the system" when searching for ...
Symptom. You are configured BI monitoring and when searching for a BW Process Chain you get the error "Error while fetching jobs from...
Read more >
opencv-cvat/dev - Gitter
I uploaded some photos using the api but I am facing the following issue: "Error during fetching a job TypeError: Cannot read property...
Read more >
Error fetching your jobs Configuration errors: 1 error occurred ...
I'm having a problem with CircleCI I can't find why it's given me that error. this is my config yml. version: 2 jobs:...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

Remember what frame you were on

Next page

Export project as dataset