Opentracing integration not working for Context-Only Spans

We have been working on integrating ElasticAPM with Lightbend’s commercial tracing solution called Cinnamon.

Cinnamon is able to use the OpenTracing bridge from ElasticAPM to integrate but has the following issues:

We have looked deeper and found a few issues that prevented Cinnamon work with Elastic APM tracer correctly.

The main issue is that Cinnamon has a special context-only propagation mode that is not supported by Elastic APM tracer.
Cinnamon uses context-only propagation in many cases to avoid creating too many spans.
And the only way to let Elastic APM tracer know of the new context is to create a new active span.

In order to achieve that Cinnamon uses a context-only spans. It's a dummy Span implementation that is never reported and used for span context propagation only.
The problem is that Elastic APM tracer can't activate such context-only spans even though it works with other tracers including Zipkin, Jaeger, or Datadog.
Elastic APM tracer expects only ApmSpan otherwise it will throw class cast exception thus we introduced the `activate-context-only-spans` setting to be able to disable active-only spans. We can't implement it on our side because it will require creating a dummy AbstractSpans that are internal Elastic APM classes and are not part of its API.

If Elastic APM supported such context-only spans then Cinnamon would be able to propagate context in context-only mode as it does with other tracers.

Cinnamon also supports auto-trace mode that creates a span every time it activates context. It will work to solve this issue but is not practical to use because it will produce too many spans for each stream stage and for all the futures. It's meant to be used for debugging purposes.

As an alternative solution Cinnamon also supports a special type of spans that we call span continuations. Basically, it's an active span that is kept unfinished while it has references to it. So, when there is no active span it uses the continuation span. It should fix the current issue with context propagation in your example but it also has some limitations and won't fix all possible problems with context propagation. So, for a better universal solution, we need Elastic APM tracer support context-only spans as described above.

Let me know if something is unclear and I can provide more information 😃

[milanvdm]

@felixbarny Sorry, I meant I was doubtful on performance on the way it currently is, not on the way that you suggested 😃

[milanvdm]

@eyalkoren I think so yes based on https://developer.lightbend.com/docs/telemetry/current/extensions/opentracing/compatibility.html

I’ll try to get more information to answer your question 😃