APM transactions from one specific service aren't shipping

Elastic Observability APM
,
1

Elastic cloud running 7.0

APM Server version:
Whichever comes with EC 7.0, which

APM Agent language and version:
npm version - elastic-apm-node v2.13.0

Issue:
Hi, we're using elastic cloud APM in our four applications. Three are sending logs fine, but the fourth is not sending transactions specifically. It is sending errors and non-transaction-related metrics just fine. I deleted the index in kibana a while ago and it was around then that it stopped getting those transactions. Could it just have never recovered ? is there a way to fix this from kibana?

Additional info: I went to the Discover page in kibana and could see everything except for the transactions from that fourth service.

Thanks guys.

2

Hi @michaelbdavidson7, welcome to the forum!

Have you already enabled trace logging? If not, I'd recommend following https://www.elastic.co/guide/en/apm/agent/nodejs/current/troubleshooting.html#debug-mode to do that, and look out for any error responses from the server. That might give us some more clues.

3

Hi Andrew,
Thanks for the reply. I did this locally, and didn't see any errors thrown. I now see that I don't think some of the APM errors are crossing over either..
What else could it be?
This used to work, I think it was from when I was messing with ES. is there a way I can just reset my ES index or something?

4

You could blow away the indices again, but I'm not convinced that'll help. I suggest checking the APM Server logs to see if there is any strange showing up there. It seems pretty odd that you would be receiving some but not all errors from the service - unless it's a short-lived service, and it's failing to flush data to the server. But then I wouldn't expect you to be seeing metrics either.

One thing that can cause data to be dropped is when labels have different types across services. If one service is recording transactions with a label foo: <number> and another service records transactions with a label foo: <string>, then the server will fail to record them for one or the other. You should see an error in the agent, though.

5

hey this was a code issue on our part, we imported our routes before the APM so the APM wasn't watching them ..
Thanks!

1 Like
6

Ahhh! Glad to hear it's sorted.

7

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.