Traces not being emitted for custom instrumentation

Trying to get our APM going with some custom and agent woven instrumentation using a GlobalTracer, running in Java 8 on undertow servlet container. Our custom code is very similar to the example code:

...
        Tracer tracer = GlobalTracer.get();

        Scope scope = tracer.buildSpan("operation-name").startActive(true);
        try {
            scope.span().setTag(DDTags.SERVICE_NAME, "my-new-service");

            // The code you're tracing
            Thread.sleep(1000);

        // If you don't call close(), the span data will NOT make it to Datadog!
        } finally {
            scope.close();
        }
...

But these custom events never appear in the APM UI, or the logs via LoggingWriter. Digging into your tracing code, it appears write() is never being called for these traces in the expireReference method.

  private void expireReference() {
    final int count = pendingReferenceCount.decrementAndGet();
    if (count == 0) {
      write();
    }
    log.debug("traceId: {} -- Expired reference. count = {}", traceId, count);
  }

because the pendingReferenceCount is always greater than 1 for custom traces, but it is 0 for agent woven instrumentation. Its not clear to me what this counter is aiming to achieve or how / when it gets incremented. Is this style of tracing (custom and auto agent) supported? Are there any gotchas? From the docs it looks like I only have to make sure to call close() on the span and things should work, as per the code close() is definitely being called, but ends up in the expireReferences method in PendingTrace.java and never gets written out.

I can see the counter gets incremented in registerContinuation and registerSpan, and we are running in a servlet container so continuations are possible, I’m not sure what to try next.

[nhoughto]

Repro test could be hard, will need to unpick it from the rest of the app 😢

I’ll put it on the list and see how hard it is, we are blocked by #429 anyway atm.

[randomanderson]

Closing as there has been no activity on this issue in over a year. Please create a new issue if the problem still persists.