Microservice tracing is not showing properly

Elastic Observability APM
1

I have a spring boot microservice project. I am using elk cloud instance for apm and kibana.
I am using consul as service discovery, rabbitMQ for message processing and webclient for service to service communication.

I run each service using this command -

java -javaagent:elk-agent/elastic-apm-agent-1.52.1.jar \
-Delastic.apm.service_name=service-name \
-Delastic.apm.secret_token=token \
-Delastic.apm.server_url=url\
-Delastic.apm.environment=local \
-Delastic.apm.universal_profiling_integration_enabled=true \
-Delastic.apm.application_packages=my-package \
-jar app.jar

I am facing two issues:

  1. There is a http request from query service to auth service. From the query service, the tracing shows the ip address of oauth instead of service name. But it works fine for ampq call through rabbitmq.

query-auth-elk
query-auth-elk1564×219 16.1 KB

  1. Another problem is, my dashboard service has a db call but in the service map, this connection is not shown. Though in the service dashboard tracing section, it detects the db call.

Do I need to add any configuration for webclient or other config?

2

I think you are hitting a limitation of what information is available at the time of the request. The http request in query doesn't know the name of the service it's sending a request to. It propagates headers, so if the oauth service were running an agent, there would be a span from it and that would resolve the service name instead of the ip.
The db call should display unless the db is something odd

3

the auth service has a running agent. I have verified it. I can see the trace of the request sent from the query service. How can I verify that the span id is sent and received properly of not?

4

hmm, you should see spans in the waterfall view, eg that one you included originally. But that pic showed just the controller and the http call. You'll need to turn your log_level to debug for the various services and track traces through to get a clearer picture of what is happening. Look for startTransaction, endTransaction, startSpan, endSpan. Filtering on a trace-id is helpful to cut down on the number of log lines to analyze