Running Django 3 with asgi fails

Describe the bug:

Django 3 supports running with asgi. APM does not work when using asgi because an issue with the format of headers in the ASGI. In ASGI the headers are a list where APM current expect it to be a dictionary.

To Reproduce

1. Create a new Django 3 project.
2. Run uvicorn pointing at newapp.asgi
3. Get stacktrace:

[2020-02-25 19:16:15 +0000] [255] [ERROR] Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/uvicorn/protocols/http/httptools_impl.py", line 385, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/usr/local/lib/python3.6/dist-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.6/dist-packages/myproject/asyncio.py", line 56, in __call__
    return await self.application(scope, receive, send)
  File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/asgi.py", line 155, in __call__
    await sync_to_async(signals.request_started.send)(sender=self.__class__, scope=scope)
  File "/usr/local/lib/python3.6/dist-packages/asgiref/sync.py", line 244, in __call__
    return await asyncio.wait_for(future, timeout=None)
  File "/usr/lib/python3.6/asyncio/tasks.py", line 339, in wait_for
    return (yield from fut)
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.6/dist-packages/asgiref/sync.py", line 277, in thread_handler
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/contextvars/__init__.py", line 38, in run
    return callable(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/django/dispatch/dispatcher.py", line 175, in send
    for receiver in self._live_receivers(sender)
  File "/usr/local/lib/python3.6/dist-packages/django/dispatch/dispatcher.py", line 175, in <listcomp>
    for receiver in self._live_receivers(sender)
  File "/usr/local/lib/python3.6/dist-packages/elasticapm/contrib/django/apps.py", line 144, in _request_started_handler
    trace_parent = TraceParent.from_headers(kwargs["scope"]["headers"])
  File "/usr/local/lib/python3.6/dist-packages/elasticapm/utils/disttracing.py", line 96, in from_headers
    tracestate = cls.merge_duplicate_headers(headers, tracestate_header_name)
  File "/usr/local/lib/python3.6/dist-packages/elasticapm/utils/disttracing.py", line 121, in merge_duplicate_headers
    return headers.get(key)
AttributeError: 'list' object has no attribute 'get'
127.0.0.1:52032 - "GET /healthz HTTP/1.0" 500

Expected behavior:

Spans are sent to APM server.

Environment (please complete the following information)

• OS: Ubuntu 18.04 (Docker)
• Python version: 3.6
• Framework and version: Django 3.0.1
• APM Server version: 7.6.0
• Agent version: 5.4.2

Additional context

Currently running under gunicorn in production but running locally with uvicorn gets the same result:

"""Settings for running gunicorn."""

bind = "0.0.0.0:8001"
workers = 4
worker_class = "uvicorn.workers.UvicornWorker"

accesslog = "-"
errorlog = "-"
loglevel = "info"

user = "www-data"
group = "www-data"

secure_scheme_headers = {"X-FORWARDED-PROTOCOL": "ssl", "X-FORWARDED-PROTO": "https"}
forwarded_allow_ips = "*"

[basepi]

My fix to asgiref has been merged, and after a bunch more experimentation, I think we’re going to rely on that fix rather than trying to hack our way around it. None of the solutions which I came up with were ideal, and as soon as the new version of asgiref is released it should be pretty trivial to upgrade.

If you want to test to make sure my fix works, you’ll want to install asgiref from its master branch, and also install contextvars (since it’s not included in python 3.6).

[blakerouse]

I added a workaround in Django settings file:

from elasticapm.utils import disttracing

# Fix disttracing.TraceParent from elaticapm
def asgi_merge_duplicate_headers(headers, key):
    if isinstance(headers, list):
        return ",".join([
            item[1]
            for item in headers
            if item[0] == key
        ])
    return headers.get(key)

disttracing.TraceParent.merge_duplicate_headers = asgi_merge_duplicate_headers