question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[tracing] Separate span creation from header propagation

See original GitHub issue

Note: This description has been rewritten to reflect changes since it was first created. The original version is included below for context.

Background:

Faced with an outgoing HTTP request, our SDKs have to make two decisions when tracing is enabled:

  • Should a span be created to track this request? and
  • Should sentry-trace and baggage headers be added to it?

Right now, we have three different options which affect the answers to those questions, and they behave differently in node and browser. The ultimate goal is to have consistent options across platforms, and - as much as possible - have each option only affect the things it says it affects.

Browser:

Currently, for browser, there are two options, tracingOrigins and shouldCreateSpanForRequest, and the behavior is as follows:

                                               | `tracingOrigins` match               | no `tracingOrigins` match            |
|----------------------------------------------|--------------------------------------|--------------------------------------|
| `shouldCreateSpanForRequest` returns `true`  | headers attached, span created       | no headers attached, no span created |
| `shouldCreateSpanForRequest` returns `false` | no headers attached, no span created | no headers attached, no span created |
| `shouldCreateSpanForRequest` is undefined    | headers attached, span created       | no headers attached, no span created |

The top right and bottom right cells are wrong - a span should be created, but it currently isn’t. In other words, tracingOrigins is controlling span creation when it shouldn’t. (It might appear that the middle left cell is also wrong - that headers should be attached in that case, because there’s a tracingOrigins match - but headers without a parent span id make no sense, because their whole purpose is to help link transactions within a trace, so it is in fact correct.)

Because fixing the behavior of tracingOrigins would be a breaking change, and because its name has always been unnecessarily confusing (in tech speak, “origins” = “web domains,” but in English, “origins” = “where things come from,” which makes no sense in the context of outgoing requests), we decided to deprecate rather than fix it, in favor of introducing a new tracePropagationTargets option with the correct behavior.

So the changes needed on the browser side are:

Node:

Currently, for node, there is one option, tracePropagationTargets, and the behavior is as follows:

| `tracePropagationTargets` match      | no `tracePropagationTargets` match   |
|--------------------------------------|--------------------------------------|
| headers attached, span created       | no headers attached, span created    |

So the changes needed on the node side are:

Ultimate Goal:

In the end, in both browser and node, the table should look like this:

                                               | `tracePropagationTargets` match      | no `tracePropagationTargets` match   |
|----------------------------------------------|--------------------------------------|--------------------------------------|
| `shouldCreateSpanForRequest` returns `true`  | headers attached, span created       | no headers attached, span created    |
| `shouldCreateSpanForRequest` returns `false` | no headers attached, no span created | no headers attached, no span created |
| `shouldCreateSpanForRequest` is undefined    | headers attached, span created       | no headers attached, span created    |

In browser, tracePropagationTargets should default to ['localhost', /^\//], whereas in node it should default to ['.*'].

While we’re here, it might be nice to unify the implementation, both between options and between platforms, so that either

  • both options have a default value on both platforms, and the undefined case therefore never needs to be handled at runtime, or
  • neither option has a default value on either platform, and the undefined case is always handled (and the default value applied) at runtime.

Potentially related to https://github.com/getsentry/develop/issues/611.


Original version of this issue, now outdated:

Click to expand

~Note: Whether or not we actually implement this change immediately, we need to make a decision here quite soon, before other SDKs start implementing this.~ It turns out no other SDKs had these two concerns intertwined, and therefore the changes coming down the pike at that point fortunately didn’t include trying to replicate this broken behavior. So this is just an issue for the JS SDK.

Current Situation:

Currently, the tracingOrigins and shouldCreateSpanForRequest options for the BrowserTracing integration control two things about outgoing http requests, based on the request destination: whether to create a span, and whether to send tracing data in the sentry-trace and baggage headers. (The difference between the two is that shouldCrateSpanForRequest is stricter - it always filters out requests filtered by tracingOrigins, but can also filter out others.)

The problem with this is that there may be outgoing requests which should be represented by a span, but which shouldn’t have headers attached (for CORS reasons, for example), and right now that’s not possible. We should divorce these two concerns, and allow those decisions to be made separately by the SDK, based on two different options set by the user. We should also bring this functionality to the Node SDK, where nothing similar currently exists.

Known constraint:

We don’t want span creation and header attachment be wholly independent, because span creation makes sense as a prerequisite for header attachment. (The alternative is a situation where it’s possible to propagate headers but not create a span. In that case, the headers would have no parent span to use, and that would break the link between the parent transaction making the http request and the child transaction handling that request.)

Proposal:

- Add a shouldAttachTracingHeadersToRequest option, which will allow control over which outgoing requests traced with a span should include tracing headers. - Make it clear that shouldCreateSpanForRequest, while it has an influence on header attachment, is not actually for controlling headers. (The default would be to attach headers to any outgoing request for which there’s a span, but that would be a default for shouldAttachTracingHeadersToRequest behavior, not for shouldCreateSpanForRequest behavior.) - Think about deprecating tracingOrigins, because _- it’s not clear which of these concerns it’s meant to address, and _ - though I get where the name comes from (“origin” = “domain” in tech-speak), as a way to filter on destination, any name involving “origin” is awfully confusing, given that in regular English, “origin” = where something comes from, not where it’s going. - Make all of this work in @sentry/node as well as @sentry/browser.

Issue Analytics

  • State:closed
  • Created a year ago
  • Reactions:1
  • Comments:12 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
timfishcommented, Nov 10, 2022

For node.js:

  • Ensure both options are documented
  • In v8, consider moving both options into the Http integration

I was just about to document these additions.

How about:

  • Add these settings to the Http integration now
  • Deprecate them in BaseNodeOptions
  • Prefer the integrations options but fallback to BaseNodeOptions so it not a breaking change
  • Add docs for these as integration options

This would get the changes in motion and mean not doing the docs changes twice.

1reaction
lobsterkatiecommented, Oct 27, 2022

Welp. We should have moved those tests after all - it would have caught https://github.com/getsentry/sentry-javascript/issues/6077, which comes from the fact that we released a new version in between merging https://github.com/getsentry/sentry-javascript/pull/6039 and https://github.com/getsentry/sentry-javascript/pull/6041.

I’ll pull the part that’s the fix out so we can release it tomorrow and leave https://github.com/getsentry/sentry-javascript/pull/6041 as purely a “add the new option” PR.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Trace Context - W3C
At a minimum they MUST propagate the traceparent and tracestate headers and guarantee traces are not broken. This behavior is also referred to ......
Read more >
Trace my mesh (part 1/3) - Medium
Envoy documents which headers are necessary to propagate depending on the tracing backend; in the case of Jaeger, use B3 trace headers, ...
Read more >
Traces | OpenTelemetry
It serializes or deserializes Span Context and provides the relevant Trace information to be propagated from one service to another.
Read more >
5. Propagation - Spring Cloud
The most common propagation approach is to copy a trace context from a client by sending an RPC request to a server receiving...
Read more >
OpenTelemetry Distributed Tracing [complete guide] - Uptrace
A span context that propagates trace ID and other data between different services. Trace is a tree of spans that shows the path...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found