DAG raises error when passing non serializable JSON object via trigger
See original GitHub issueWhen passing a non serializable JSON object in a trigger, I get the following error below. The logs become unavailable.
my code:
task_trigger_ad_attribution = TriggerDagRunOperator(
task_id='trigger_ad_attribution',
trigger_dag_id=AD_ATTRIBUTION_DAG_ID,
conf={"message": "Triggered from display trigger",
'trigger_info':
{'dag_id':DAG_ID,
'now':datetime.datetime.now(),
},
'trigger_date' : '{{execution_date}}'
},
)
Ooops!
Something bad has happened.
Please consider letting us know by creating a bug report using GitHub.
Python version: 3.6.9
Airflow version: 2.0.0
Node: henry-Inspiron-5566
-------------------------------------------------------------------------------
Traceback (most recent call last):
File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/airflow/www/auth.py", line 34, in decorated
return func(*args, **kwargs)
File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/airflow/www/decorators.py", line 97, in view_func
return f(*args, **kwargs)
File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/airflow/www/decorators.py", line 60, in wrapper
return f(*args, **kwargs)
File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/airflow/www/views.py", line 1997, in tree
data = htmlsafe_json_dumps(data, separators=(',', ':'))
File "/home/henry/Envs2/airflow3/lib/python3.6/site-packages/jinja2/utils.py", line 614, in htmlsafe_json_dumps
dumper(obj, **kwargs)
File "/usr/lib/python3.6/json/__init__.py", line 238, in dumps
**kw).encode(obj)
File "/usr/lib/python3.6/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python3.6/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/usr/lib/python3.6/json/encoder.py", line 180, in default
o.__class__.__name__)
TypeError: Object of type 'datetime' is not JSON serializable
Issue Analytics
- State:
- Created 3 years ago
- Comments:9 (4 by maintainers)
Top Results From Across the Web
Can not use celery delay for saved form: object is not JSON ...
Task's argument should be serializable (i.e. string, int, etc.). To fix error you can pass topic_id as argument and fetch topic object ......
Read more >Best Practices - Apache Airflow
This test should ensure that your DAG does not contain a piece of code that raises error while loading. No additional code needs...
Read more >[GitHub] [airflow] turbaszek opened a new issue #16283
**Description** Make XComArg JSON-serializable so user can use ... from airflow.operators.trigger_dagrun import TriggerDagRunOperator from ...
Read more >Airflow 2.0: DAG Authoring Redesigned - Medium
It means that it can be passed to either a normal operator or to ... The data has to be small and json-serializable...
Read more >7 Common Errors to Check When Debugging Airflow DAGs
Check out the list of common Airflow deployment errors, ... Note: You can manually trigger a DAG run via Airflow's UI directly on...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Currently, we don’t support it as we need to de-serialize it for Webserver and show it in the UI in List DagRun page. Needs a bigger change to support other objects that are not JSON serializable atm
We need this feature because previous we can get over it by passing a python_callable and now it is not possible.