question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Missing traces/spans for requests

See original GitHub issue

We have a third party docker containers were we have injected the opentelemetry java agent. That works great. Metrics, logging are extracted.

But not for all requests (actual the requests I want to know more about), are not traced.

This is information from the metrics (prometheus endpoint): Irrelevant information is stripped

http_server_duration_count{http_flavor="1.1",http_host="internal_hostname:8081",http_method="POST",http_scheme="http",http_status_code="200",service_name="myservice"} 1366 1650014515741
http_server_duration_count{http_flavor="1.1",http_host="x_internal_ip.x.x.x:8081",http_method="POST",http_route="/api/v1/aangiftes/berekening",http_scheme="http",http_status_code="500",service_name="myservice"} 1 1650014515741
http_server_duration_count{http_flavor="1.1",http_host="localhost:8081",http_method="OPTIONS",http_route="/*",http_scheme="http",http_status_code="202",service_name="myservice"} 1187 1650014515741
http_server_duration_count{http_flavor="1.1",http_host="internal_hostname:8081",http_method="POST",http_route="/api/v1/aangiftes/berekening",http_scheme="http",http_status_code="500",service_name="myservice"} 1 1650014515741
otlp_exporter_exported{service_name="myservice",success="true",type="span"} 1191 1650014515741
otlp_exporter_seen{service_name="myservice",type="span"} 1191 1650014515741
processedSpans{dropped="false",service_name="myservice",spanProcessorType="BatchSpanProcessor"} 1191 1650014515741

I see a number of requests without an http_route. These requests do not get traced. All the other requests are send either curl or by the monitoring (the OPTIONS requests). These requests do get traced.

The third party Java service is using Spring boot with Jersey JaxRS.

The invocation of these services is done by .Net code in Azure. HttpClient postAsync with a payload. These are also send to /api/v1/aangiftes/berekening, but apparently not recognized for that http_route.

My question is: How can we get the metrics right, then it also gets the http_route and how do we get the traces for these requests as well.

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:11 (5 by maintainers)

github_iconTop GitHub Comments

2reactions
cboscommented, Apr 19, 2022

@trask @mateuszrzeszutek Thanks, I just started using 1.13.0 and the issue is solved!

0reactions
traskcommented, Apr 17, 2022

hi @cbos! current target for 1.13.0 release is Monday.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Method: projects.traces.spans.createSpan - Google Cloud
Request body. The request body contains data with the following structure: ... If set, allows implementation to detect missing child spans.
Read more >
root span does not reach collector when running under uwsgi
If I run my app using werkzeug.serving.run_simple all spans are propagated just fine and my traces/spans are error free.
Read more >
Untitled
[SPAN_ID] is a unique identifier for a span within a trace; it is a 16-character hexadecimal encoding of an 8-byte array. (required) body:...
Read more >
Getting traces/spans from a LoadBalancer : r/googlecloud - Reddit
Is it possible to collect and export traces/spans from L7 LBs, ... There is a X-Cloud-Trace-Context request header passed from LBs into backends, ......
Read more >
googleapis documentation - Google Cloud
Interfaces; Params$Resource$Projects$Traces$Spans$Createspan. Info; Source ... requestBody: Schema$Span. Type : Schema$Span. Optional. Request body metadata ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found