Service Name is still showing after deletion in APM

Elastic Observability APM
,
1

Hi team,

I have deleted a service name in APM using below query from console, but still after few minutes the service name again pop-ups in APM section. I even tried to delete all the apm indexes from Management section, (also restarted apm-server), but still after few minutes service name pop-ups there in APM section.

POST apm-7.7.0-*/_delete_by_query
{
  "query": {
     "term": {
       "service.name": "my-App"
     }
  }
}

I have attached the apm-agent using apm-agent-attach-1.16.0-standalone.jar and currently it is not attached to my java application.

Can you please advise how to fix this issue permanently. I have tried everything but nothing worked.

Below are the details of my setup.

Kibana version : 7.7.0

Elasticsearch version : 7.7.0

APM Server version : 7.7.0

APM Agent language and version : Java and apm-agent-attach-1.16.0-standalone.jar

Browser version : Chrome Version 83.0.4103.61

Original install method (e.g. download page, yum, deb, from source, etc.) and version : Yum

Fresh install or upgraded from other version? : Upgraded from 7.3.2

Is there anything special in your setup? No Load balancer used for APM server

Br
Pankaj N

2

Hi there,

If I'm understanding this correctly, after you run _delete_by_query the service disappears but then it shows up again after a couple of minutes?
If this is the case it sounds very much like the service is still running and sending data. Can you double-check whether this is the case?

Another reason might be that you have apm data from other versions than 7.7. You can delete across all versions with POST apm-*-*/_delete_by_query

Let me know how it goes.

3

Hi @sqren

Thanks for your reply.

Yes your understanding is correct, service disappears after running the query but re-appears after few minutes.

I have used the apm-agent-attach-1.16.0-standalone.jar method for the apm-agent with my application and i have removed it by giving a stop/start to my application. So, there is no chance apm-agent will automatically get attached to the application, as i was attaching the apm-agent imanually only. Also, i have verified it, as no data is getting showed up in the APM section. Only the service name re-appears.

I am also confirming that there is no apm data from other versions than 7.7

Br
Pankaj N

4

Normally, the service name can only be shown if there's data for that service. Could you search for documents of that service in the discover tab? Maybe there's another instance of that service running somewhere that reports metrics in the background. In that case, you wouldn't see transactions in the UI as there are only metrics. Maybe you accidentally attached the agent to multiple services. Looking at the metadata of the metric events, you should be able to identify the process that's still sending metrics.

5

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.