question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Distributed tracing not working with elastic APM using Netflix Eureka registry server

See original GitHub issue

Distributed tracing not working with elastic APM using Netflix Eureka registry server

We follow micro service architecture where all the services registered it self in Netflix eureka registry server, in-order to make inter-service call with each other. All the micro services are springboot projects. we have installed elastic search( version : 6.6.0), kibana (version : 6.6.0), elastic APM (version : 6.6.0) and we started all the springboot application using elastic-apm-agent (version : 1.3.0) java agent.

When one service makes call to other service the trace is not being captured only the calling service SQL query performance is getting captured.

To Reproduce Steps to reproduce the behavior:

  1. Create a P0 (spring boot application) to Enable EurekaServer (which acts like discovery server)
  2. Created P1 and P2 (spring boot application) registered with eureka discover service.
  3. Installed elastic search (version : 6.6.0),kibana (version : 6.6.0), elstic APM (version : 6.6.0) in different servers made all the configurations and started all of them.
  4. Started P0,P1,P2 with elastic-apm java agent ( version : 1.3.0).
  5. Created an Rest API in P1, That makes a call to P2’s Rest API using @FeignClient.

API created an (ie A1) in P1 will do some CRUD operations then makes call to P2 API (ie, A2).

Error: We are able to see the SQL query performance of A1 in in elastic APM but we are unable to see A1->A2 call trace in APM

Expected behavior Time taken buy the A1-> A2 call should be shown in the APM console

Debug logs Attach your debug logs. See the documentation about how to enable debug logging.

Click to expand

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:8 (4 by maintainers)

github_iconTop GitHub Comments

2reactions
Harishchanticommented, Mar 6, 2019

@eyalkoren , Sorry for the delayed response. I was caught up with multiple tasks.

Feign client internally uses okhttpclient. Distributed tracing is working properly with elastic-apm-agent-1.4.0.jar .

Thanks for the support.

1reaction
eyalkorencommented, Mar 3, 2019

Looks like there was a failure sending data to the APM server ("server-side-error.log). A full log would be better and APM server log can be useful as well. Note that most stack traces you see now are not errors but related to the fact you set the log level to TRACE.

Did you try to figure out what your Feign client is using for making the actual HTTP requests? Is there any configuration you made that may indicate that? This is very important information for analysing distributed tracing issues.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Distributed tracing | APM User Guide [8.5]
A trace is a group of transactions and spans with a common root. Each trace tracks the entirety of a single request. When...
Read more >
Open Source Application Performance Monitoring (APM)
OpenTracing wants to become something like a standard tracing API/library (such as SLF4J for logging in Java) which is already build in in ......
Read more >
How to Set Up Distributed Tracing in Microservices with ...
Have Basic understanding of how to set up a microservice using spring boot and spring cloud. Install the zipkin server; Install Elasticsearch, ...
Read more >
Tracing and Tracing based Metrics Analyze Plugins
The following plugins provide the distributed tracing capability, and the OAP backend would analyze the topology and metrics based on the tracing data....
Read more >
Profile for Elastic Inc.
How to instrument a polyglot microservices application with Elastic APM ... application using Elastic APM, with a focus on distributed tracing.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found