JSON notation in Airflow DAG comments causing KeyError

–> Apache Airflow version: 1.10.11 (code is working fine in 1.10.10)

Environment: Python 3.7

• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release): Catalina 10.15.5
• Kernel (e.g. uname -a):
• Install tools:
• Others:

What happened:

KeyError: ‘“randomKey”’

What you expected to happen:

Expected the block comment on the top of the DAG to be copied as is into the DAG description box in the UI

How to reproduce it: Add the following code to the top of your DAG

“”" This is a test dag. this is the description of it. This is how you can trigger it with CLI.

airflow trigger_dag --conf ‘{“randomKey”:30}’ airflow-test-dag

–conf options: randomKey:<INT> - Optional

“”"

Anything else we need to know: Works fine with Airflow 1.10.10 and Python 3.7.

** UI Logs: **

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.7/site-packages/flask_admin/base.py", line 69, in inner
    return self._run_view(f, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/flask_admin/base.py", line 368, in _run_view
    return fn(self, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/flask_login/utils.py", line 258, in decorated_view
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/airflow/www/utils.py", line 380, in view_func
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/airflow/www/utils.py", line 286, in wrapper
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py", line 74, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/airflow/www/views.py", line 1749, in graph
    doc_md = wrapped_markdown(getattr(dag, 'doc_md', None), css_class='dag-doc')
  File "/usr/local/lib/python3.7/site-packages/airflow/www/views.py", line 258, in wrapped_markdown
    ).format(css_class=css_class)
  File "/usr/local/lib/python3.7/site-packages/markupsafe/__init__.py", line 213, in format
    return self.__class__(formatter.vformat(self, args, kwargs))
  File "/usr/local/Cellar/python/3.7.8/Frameworks/Python.framework/Versions/3.7/lib/python3.7/string.py", line 190, in vformat
    result, _ = self._vformat(format_string, args, kwargs, used_args, 2)
  File "/usr/local/Cellar/python/3.7.8/Frameworks/Python.framework/Versions/3.7/lib/python3.7/string.py", line 230, in _vformat
    obj, arg_used = self.get_field(field_name, args, kwargs)
  File "/usr/local/Cellar/python/3.7.8/Frameworks/Python.framework/Versions/3.7/lib/python3.7/string.py", line 295, in get_field
    obj = self.get_value(first, args, kwargs)
  File "/usr/local/Cellar/python/3.7.8/Frameworks/Python.framework/Versions/3.7/lib/python3.7/string.py", line 252, in get_value
    return kwargs[key]
  File "/usr/local/lib/python3.7/site-packages/markupsafe/__init__.py", line 249, in __getitem__
    return self._kwargs[key]
KeyError: '"randomKey"'

[kaxil]

I was able to reproduce it but looks like it can be easily fixed if you remove the following from docstrings from the DAG (https://github.com/teamclairvoyant/airflow-maintenance-dags/blob/63ddbfb19d53fa7b44343dc0525cf1b60f16fed6/db-cleanup/airflow-db-cleanup.py#L6-L9):

airflow trigger_dag --conf '{"maxDBEntryAgeInDays":30}' airflow-db-cleanup

[potiuk]

Right I know what it is and the problem is also in master. Fix is coming.