Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Revisit use of read_namespaced_pod_log

See original GitHub issue

Right now we use the k8s read_namespaced_pod_log to extract the GraphQL response printed by dagster-graphql to stdout—see here

We should probably find a less fragile way to accomplish this; it’s not clear to me that this API is designed for durability / reliable reads of pod outputs

Issue Analytics

State:
Created 4 years ago
Comments:8 (8 by maintainers)

Top GitHub Comments

1reaction

natekuppcommented, Jul 28, 2020

per conversation this morning with @catherinewu, we could instead just have the containers write directly to the instance database and remove this parse_raw_log_lines entirely. If the containers had access to the instance, we could also remove passing tags over the graphql interface; see https://dagster.phacility.com/D4017

0reactions

johannkmcommented, Mar 5, 2022

We’ve solved this with the k8s_job_executor, it persists with the celery_k8s_job_executor