APM Agent data not getting connected/uploaded to APM Server

Stud21 · April 16, 2021, 2:41am

I unable to setup APM Agent load data to get updated into APM Server.
In Kibana APM Server status says 'You have correctly setup APM Server'
However when I click on the Agent status in Kibana it says 'No data has been received from agents yet'.
However if I go back to APM services, it does show my application listed with Name, Avg. response, Trans. per minute and Error rate %.

I have tried to run the application as stated in Kibana as under:
sudo java -javaagent:elastic-apm-agent-1.22.0.jar
-Delastic.apm.service_name=petclinic-java
-Delastic.apm.server_urls=https://xyz.com.au:8200
-Delastic.apm.secret_token=
-Delastic.apm.application_packages=co.elastic.apm.petclient
-jar spring-petclinic-2.4.2.jar
This enables my application and I am able to access, view and use it using a web browser

Looking at the Java application log it shows errors as below:
'2021-04-16 12:01:08.170 INFO 2892 --- [ main] o.s.s.petclinic.PetClinicApplication : Started PetClinicApplication in 12.433 seconds (JVM running for 20.207)
2021-04-16 12:01:24,168 [elastic-apm-server-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Error trying to connect to APM Server. Some details about SSL configurations corresponding the current connection are logged at INFO level.
2021-04-16 12:01:24,169 [elastic-apm-server-reporter] INFO co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Default cipher suites: [TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]'

Following is the error logged in APM logs:
'2021-04-15T17:08:17.110+1000 DEBUG [kibana] kibana/connecting_client.go:74 Trying to obtain connection to Kibana.
2021-04-15T17:08:17.110+1000 INFO kibana/client.go:119 Kibana url: https://xyz.com.au:5601
2021-04-15T17:08:17.124+1000 ERROR [kibana] kibana/connecting_client.go:77 failed to obtain connection to Kibana: fail to get the Kibana version: HTTP GET request to https://xyz.com.au:5601/api/status fails: . Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}.
2021-04-15T17:08:17.282+1000 DEBUG [kibana] kibana/connecting_client.go:74 Trying to obtain connection to Kibana.'

My query is:
Do I need to enable SSL connection between APM server and agent? How?
If APM server status show green and under APM Services the name correctly appears as petclinic-java then what is stopping the data update

Sylvain_Juge · April 19, 2021, 12:59pm

Hi @Stud21 ,

There are two things not related here:

communication between apm-agent (Java) and apm-server
communication between apm-server and Kibana

For (1) you seem to be using an https URL, which implies that SSL/TLS is being used to reach to apm-server, if it's a local instance without SSL/TLS enabled you could just try with http instead.

For (2) you get an HTTP 401 response code, which means it's not related to SSL/TLS, but more about authorization between apm-server and Kibana.

Stud21 · April 19, 2021, 11:34pm

Hi Sylvain,
I have tried running the java (petclinic) application as under:
sudo java -javaagent:elastic-apm-agent-1.22.0.jar
-Delastic.apm.service_name=petclinic-java
-Delastic.apm.server_urls=http://xyz.com.au:8200
-Delastic.apm.secret_token=
-Delastic.apm.application_packages=co.elastic.apm.petclient
-jar spring-petclinic-2.4.2.jar

I have verified and find that APM server is working fine with indexes running and APM Server status stating 'You have correctly setup APM Server'.

I have attached the java app output file here.

Conclusion:
In Kibana, APM agent status still says 'No data has been received from agents yet' and I'm unable to locate any services registered or data in APM

system · May 10, 2021, 7:35pm

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.