question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

DAG raises error when passing non serializable JSON object via trigger

See original GitHub issue

When passing a non serializable JSON object in a trigger, I get the following error below. The logs become unavailable.

my code:

task_trigger_ad_attribution = TriggerDagRunOperator(
             task_id='trigger_ad_attribution',
             trigger_dag_id=AD_ATTRIBUTION_DAG_ID,
              conf={"message": "Triggered from display trigger",
                  'trigger_info':
                     {'dag_id':DAG_ID,
                         'now':datetime.datetime.now(),
                         },
                  'trigger_date' : '{{execution_date}}'
                  },
             )
Ooops!
Something bad has happened.
Please consider letting us know by creating a bug report using GitHub.

Python version: 3.6.9
Airflow version: 2.0.0
Node: henry-Inspiron-5566
-------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/airflow/www/auth.py", line 34, in decorated
    return func(*args, **kwargs)
  File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/airflow/www/decorators.py", line 97, in view_func
    return f(*args, **kwargs)
  File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/airflow/www/decorators.py", line 60, in wrapper
    return f(*args, **kwargs)
  File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/airflow/www/views.py", line 1997, in tree
    data = htmlsafe_json_dumps(data, separators=(',', ':'))
  File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/jinja2/utils.py", line 614, in htmlsafe_json_dumps
    dumper(obj, **kwargs)
  File "/usr/lib/python3.6/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/usr/lib/python3.6/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.6/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python3.6/json/encoder.py", line 180, in default
    o.__class__.__name__)
TypeError: Object of type 'datetime' is not JSON serializable

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:9 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
kaxilcommented, Feb 10, 2021

Currently, we don’t support it as we need to de-serialize it for Webserver and show it in the UI in List DagRun page. Needs a bigger change to support other objects that are not JSON serializable atm

0reactions
wuchichungcommented, Sep 15, 2021

We need this feature because previous we can get over it by passing a python_callable and now it is not possible.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Can not use celery delay for saved form: object is not JSON ...
Task's argument should be serializable (i.e. string, int, etc.). To fix error you can pass topic_id as argument and fetch topic object ......
Read more >
Best Practices - Apache Airflow
This test should ensure that your DAG does not contain a piece of code that raises error while loading. No additional code needs...
Read more >
[GitHub] [airflow] turbaszek opened a new issue #16283
**Description** Make XComArg JSON-serializable so user can use ... from airflow.operators.trigger_dagrun import TriggerDagRunOperator from ...
Read more >
Airflow 2.0: DAG Authoring Redesigned - Medium
It means that it can be passed to either a normal operator or to ... The data has to be small and json-serializable...
Read more >
7 Common Errors to Check When Debugging Airflow DAGs
Check out the list of common Airflow deployment errors, ... Note: You can manually trigger a DAG run via Airflow's UI directly on...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found