Stable API task logs

Apache Airflow version: 2.0.0

Kubernetes version (if you are using kubernetes) (use kubectl version): NA

Environment:

• Cloud provider or hardware configuration: PC (docker-compose)
• OS (e.g. from /etc/os-release): Linux mint 20 (for PC), Debian Buster in container
• Kernel (e.g. uname -a): Linux 607a1bfeebd2 5.4.0-60-generic #67-Ubuntu SMP Tue Jan 5 18:31:36 UTC 2021 x86_64 GNU/Linux
• Install tools: Poetry (so pipy)
• Others: Using python 3.8.6, with Celery Executor, one worker Task did run properly

What happened: I tried to get the logs of a task instance using the stable Rest API through the Swagger UI included in Airflow, and it crashed (got a stack trace) I got 500 error

engine-webserver_1     | 2021-01-12T16:45:18.465370280Z [2021-01-12 16:45:18,464] {app.py:1891} ERROR - Exception on /api/v1/dags/insert/dagRuns/manual__2021-01-12T15:05:59.560500+00:00/taskInstances/insert-db/logs/0 [GET]
engine-webserver_1     | 2021-01-12T16:45:18.465391147Z Traceback (most recent call last):
engine-webserver_1     | 2021-01-12T16:45:18.465394643Z   File "/brain/engine/.cache/poetry/meta-vSi4r4R8-py3.8/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
engine-webserver_1     | 2021-01-12T16:45:18.465397709Z     response = self.full_dispatch_request()
engine-webserver_1     | 2021-01-12T16:45:18.465400161Z   File "/brain/engine/.cache/poetry/meta-vSi4r4R8-py3.8/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
engine-webserver_1     | 2021-01-12T16:45:18.465402912Z     rv = self.handle_user_exception(e)
engine-webserver_1     | 2021-01-12T16:45:18.465405405Z   File "/brain/engine/.cache/poetry/meta-vSi4r4R8-py3.8/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
engine-webserver_1     | 2021-01-12T16:45:18.465407715Z     reraise(exc_type, exc_value, tb)
engine-webserver_1     | 2021-01-12T16:45:18.465409739Z   File "/brain/engine/.cache/poetry/meta-vSi4r4R8-py3.8/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
engine-webserver_1     | 2021-01-12T16:45:18.465412258Z     raise value
engine-webserver_1     | 2021-01-12T16:45:18.465414560Z   File "/brain/engine/.cache/poetry/meta-vSi4r4R8-py3.8/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
engine-webserver_1     | 2021-01-12T16:45:18.465425555Z     rv = self.dispatch_request()
engine-webserver_1     | 2021-01-12T16:45:18.465427999Z   File "/brain/engine/.cache/poetry/meta-vSi4r4R8-py3.8/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
engine-webserver_1     | 2021-01-12T16:45:18.465429697Z     return self.view_functions[rule.endpoint](**req.view_args)
engine-webserver_1     | 2021-01-12T16:45:18.465431146Z   File "/brain/engine/.cache/poetry/meta-vSi4r4R8-py3.8/lib/python3.8/site-packages/connexion/decorators/decorator.py", line 48, in wrapper
engine-webserver_1     | 2021-01-12T16:45:18.465433001Z     response = function(request)
engine-webserver_1     | 2021-01-12T16:45:18.465434308Z   File "/brain/engine/.cache/poetry/meta-vSi4r4R8-py3.8/lib/python3.8/site-packages/connexion/decorators/uri_parsing.py", line 144, in wrapper
engine-webserver_1     | 2021-01-12T16:45:18.465435841Z     response = function(request)
engine-webserver_1     | 2021-01-12T16:45:18.465437122Z   File "/brain/engine/.cache/poetry/meta-vSi4r4R8-py3.8/lib/python3.8/site-packages/connexion/decorators/validation.py", line 384, in wrapper
engine-webserver_1     | 2021-01-12T16:45:18.465438620Z     return function(request)
engine-webserver_1     | 2021-01-12T16:45:18.465440074Z   File "/brain/engine/.cache/poetry/meta-vSi4r4R8-py3.8/lib/python3.8/site-packages/connexion/decorators/response.py", line 103, in wrapper
engine-webserver_1     | 2021-01-12T16:45:18.465441667Z     response = function(request)
engine-webserver_1     | 2021-01-12T16:45:18.465443086Z   File "/brain/engine/.cache/poetry/meta-vSi4r4R8-py3.8/lib/python3.8/site-packages/connexion/decorators/parameter.py", line 121, in wrapper
engine-webserver_1     | 2021-01-12T16:45:18.465445345Z     return function(**kwargs)
engine-webserver_1     | 2021-01-12T16:45:18.465446713Z   File "/brain/engine/.cache/poetry/meta-vSi4r4R8-py3.8/lib/python3.8/site-packages/airflow/api_connexion/security.py", line 47, in decorated
engine-webserver_1     | 2021-01-12T16:45:18.465448202Z     return func(*args, **kwargs)
engine-webserver_1     | 2021-01-12T16:45:18.465449538Z   File "/brain/engine/.cache/poetry/meta-vSi4r4R8-py3.8/lib/python3.8/site-packages/airflow/utils/session.py", line 65, in wrapper
engine-webserver_1     | 2021-01-12T16:45:18.465451032Z     return func(*args, session=session, **kwargs)
engine-webserver_1     | 2021-01-12T16:45:18.465452504Z   File "/brain/engine/.cache/poetry/meta-vSi4r4R8-py3.8/lib/python3.8/site-packages/airflow/api_connexion/endpoints/log_endpoint.py", line 81, in get_log
engine-webserver_1     | 2021-01-12T16:45:18.465454135Z     logs, metadata = task_log_reader.read_log_chunks(ti, task_try_number, metadata)
engine-webserver_1     | 2021-01-12T16:45:18.465455658Z   File "/brain/engine/.cache/poetry/meta-vSi4r4R8-py3.8/lib/python3.8/site-packages/airflow/utils/log/log_reader.py", line 58, in read_log_chunks
engine-webserver_1     | 2021-01-12T16:45:18.465457226Z     logs, metadatas = self.log_handler.read(ti, try_number, metadata=metadata)
engine-webserver_1     | 2021-01-12T16:45:18.465458632Z ValueError: not enough values to unpack (expected 2, got 1)

What you expected to happen: I expected to get the logs of my task

How to reproduce it: I think it’s everytime (at least on my side)

Anything else we need to know: Other stable API call, such as getting list of dags runs, task instance, etc worked well. Logs is appearing well if I go to

EDIT : Ok, I’m stupid, I put 0 as try number, instead of 1… So not a big bug, though I think 0 as try number should be a 400 status response, not 500 crash. Should I keep it open ?

[potiuk]

Am I on right path? and do I have to add unit test for this scenario?

Definitely looks like you are 😃

and do I have to add unit test for this scenario?

It would be great to add them indeed calling a read method with different try_numbers and expecting tuple with two elements would be great

[vikramcse]

Hi @potiuk I am able to reproduce the issue. while debugging I noticed that when the try_number < 1 the read function from FileTaskHandler has only one return parameter which is logs, where there should be two return parameters logs and metadata.

To fix this issue I have added a fix, which returns two parameters instead of one

the output of above modification is

Am I on right path? and do I have to add unit test for this scenario?

thank you