APM Server not identified by Kibana

Kibana version : 7.17

Elasticsearch version : 7.17

APM Server version : 7.17

APM Agent language and version : java 1.8

Fresh install or upgraded from other version? fresh install

Is there anything special in your setup? For example, are you using the Logstash or Kafka outputs? Are you using a load balancer in front of the APM Servers? Have you changed index pattern, generated custom templates, changed agent configuration etc. :-no

I have been working with APM server and creating spans and transactions till yesterday. Suddenly today I see that Kibana does not show my APM data and server is not detected. But The URL at 8200 port gives me the below:

{
  "build_date": "2022-05-18T06:49:31Z",
  "build_sha": "4440c47d2c53b3b0b969138a395d6cb0c94b12c1",
  "publish_ready": true,
  "version": "7.17.4"
}

When I check the APM server status in Kibana it gives me:- No APM Server detected. Please make sure it is running and you have updated to 7.0 or higher.

I have this error in Elasticsearch logs

[2022-06-23T02:03:49,507][ERROR][o.e.x.i.IndexLifecycleRunner] [snsl-vm-411] policy [apm-rollover-30-days] for index [apm-7.17.4-metric] failed on step [{"phase":"hot","action":"rollover","name":"check-rollover-ready"}]. Moving to ERROR step

But this error was there for a very long time ...Why wouldnt Kibana detect the APm Server ..

Hi could somebody give me any leads

Hi @anjana1 ,
have you applied any changes (upgrades, config changes etc.) before the issues started?
You can start with looking into the apm agent and the apm server logs and identify if there is anything suspicious logged.
Please also check via discover or dev-tools if APM data are still ingested into ES and just not show up in Kibana APM UI or if in fact the ingestion stopped.

I can see span data coming into Apm server logs . But for some reason Apm app is not recognising the server .. no error in logs anywhere . Data is not injected in discover screen of kibana

Can you expand the time range in the Kibana APM UI until a date where APM data were still shown and then check if there is anything suspicious in the apm-server logs for this time?
Please also check the Kibana logs.

Hi is this issue link below relevant to ours . I also have the error

Exiting: error loading config file: config file ("/etc/apm-server/apm-server.yml") must be owned by the user identifier (uid=0) or root