question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Questions on the Distributed Trace extension

See original GitHub issue

1. Why not just use traceparent as the attribute?

There is already precedence for doing this with content-type in the broader spec. Ex, from datacontenttype:

For some binary mode protocol bindings, this field is directly mapped to the respective protocol’s content-type metadata property.

Which is specified in the HTTP binding spec

For the binary mode, the HTTP Content-Type header value corresponds to (MUST be populated from or written to) the CloudEvents datacontenttype attribute. Note that a ce-datacontenttype HTTP header MUST NOT also be present in the message.

It doesn’t feel intuitive to allow both traceparent and ce-traceparent as distinct things. Which kinda leads to my next questions:

2. The spec says …

Given a multi hop event transmission, the Distributed Tracing Extension, if used, MUST carry the trace information of the starting trace of the transmission. In other words, it MUST NOT carry trace information of each individual hop, since this information is usually carried using protocol specific headers, understood by tools like OpenTelemetry.

2.1) What’s the purpose in changing the behavior of traceparent here vs how the W3C spec says it should be? And why not rename it like “starting traceparent” or something if that’s the case?

2.2) Even if there is a good reason, the example seems to allow ignoring the original trace info (by the fact that there are two different traceIDs). Even if you were going to stick with saying traceparent MUST carry the trace information of the starting trace – I feel like other trace info should at least be on the same tree.

Ex, current example (different trace-id, different parent-id):

-H ‘ce-traceparent: 00-0af7651916cd43dd8448eb211c80319c-b9c7c989f97918e1-01’
-H ‘traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01’ \

In this case, I’d expect something like (same trace-id, different parent-id):

-H ‘ce-traceparent: 00-0af7651916cd43dd8448eb211c80319c-b9c7c989f97918e1-01’
-H ‘traceparent: 00-0af7651916cd43dd8448eb211c80319c-00f067aa0ba902b7-01’ \

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:8

github_iconTop GitHub Comments

2reactions
duglincommented, Jun 14, 2022

@joaopgrassi @ntextreme3 thanks for the good chat - lots of good info in there!

2reactions
joaopgrassicommented, Jun 10, 2022

The way I read it here at first is that App3…AppN would always receive ce-traceparent matching the “starting trace” (App1).

If the same initial event is forwarded (or travels) to/from AppN, then all of those will link to App1. The simple way to think is: subsequent apps will extract whatever context is in the event and add it as a link to its own trace. If the initial event “goes through many hops” it’s then clear all of those hops will link to the initial App1.

the “multi hop” period ends once the next instrumented app consumes the message, and the “starting” ce-traceparent doesn’t continue propagating forever?

Yes, exactly. The multi hop ends when the next consumer doesn’t forward the event anymore. It “settles” it.

In your example, when App2 receives the initial event from App1, it will start its own trace, and extract the context (ce-traceparent) from the event and add it as a link, linking App2 <> App1. Then say, the event in App2 caused some side effect and App2 needs to send a new event to App3. Then the whole process starts again - App2 creates the new event, a new span and adds it inside of it. The new event is forwarded and, at some point, App3 receives it. Then you have a link of App3 <> App2.

I did some drawings for the case above, maybe it helps

Apps interaction: image

Achieved trace, using the conventions from here/OpenTelemetry image

Of course, all other actions that happens in either apps will be part of each trace, as long there’s some auto-instrumentation in place. Say, App2 makes some Db calls and those are instrumented. It will all be part of App2’s trace.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Clarification in Distributed Tracing Extension #603 - GitHub
Couple of questions: What if an event with both representations not matching goes through a middleware non-cloudevents aware but tracing aware? What should...
Read more >
Introduction to distributed tracing - New Relic Documentation
An intro to New Relic's distributed tracing feature. ... Distributed tracing tracks and observes service requests as they flow through distributed systems.
Read more >
What is distributed tracing and why does it matter? - Dynatrace
Distributed tracing is a method of observing requests as they propagate through distributed cloud environments. Learn how it works.
Read more >
HiveMQ Enterprise Extension for Distributed Tracing
Monitor and track MQTT data end-to-end & in real-time on your APM solution with HiveMQ Distributed Tracing Extension featuring OpenTelemetry integration.
Read more >
Distributed Tracing Guide: Monitoring, Logging, Standards ...
Problems with microservice reporting · One problem with a distributed tracing strategy is that much of the detail depends on the amount of...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found