Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Unable to read log if the pod name is too long

See original GitHub issue

Apache Airflow version: 1.10.12

Kubernetes version (if you are using kubernetes) (use kubectl version): 1.15

Environment:

Cloud provider or hardware configuration: Not related
OS (e.g. from /etc/os-release): Not related
Kernel (e.g. uname -a): Not related
Install tools: Not related
Others: Not related

What happened:

We are using kubernetes executor and try to read logs from webserver directly via the Kubernetes API. (https://github.com/apache/airflow/pull/12598) We found that if the pod name is longer than 63 characters, it will be 404.

*** Unable to fetch logs from worker pod thisisaverylooooooooooooooooooooongpodname-8fba3581e4614c469655 ***
(404)
Reason: Not Found
HTTP response headers: HTTPHeaderDict({'Audit-Id': 'xxxxxxxx', 'Content-Type': 'application/json', 'Date': 'Mon, 11 Jan 2021 09:57:37 GMT', 'Content-Length': '294'})
HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods \\"thisisaverylooooooooooooooooooooongpodname-8fba3581e4614c469655\\" not found","reason":"NotFound","details":{"name":"thisisaverylooooooooooooooooooooongpodname-8fba3581e4614c469655","kind":"pods"},"code":404}\n'

What you expected to happen:

*** Trying to get logs (last 100 lines) from worker pod shortnameeeeeeeeeeeeeeetask-a11e7ef862f641e4a0a4a686482f43a4 ***

[2021-01-11 10:22:29,200] {__init__.py:50} INFO - Using executor LocalExecutor
[2021-01-11 10:22:29,200] {dagbag.py:417} INFO - Filling up the DagBag from /usr/local/airflow/dags/shortname.py

How to reproduce it: Name a dag / task with long name.

Anything else we need to know: