APM agent cannot connect to apm server

Elastic Observability APM
1

Kibana version: 7.5.2

Elasticsearch version: 7.5.2

APM Server version: 7.5.2

**APM Agent language **: java

*Original install method: ECK operator 1.0.0 GA

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):

Apm agent unable to send data to apm server

Provide logs and/or server output (if relevant):

2020-01-30 07:47:07.733 [apm-server-healthcheck] WARN co.elastic.apm.agent.report.ApmServerHealthChecker - Elastic APM server http://apm-server-quickstart-apm-http.elastic-system.svc.cluster.local:8200/ is not available (Connection refu
2020-01-30 07:47:07.733 [apm-server-healthcheck] ERROR co.elastic.apm.agent.report.HttpUtils - Exception when closing input stream of HttpURLConnection.
2020-01-30 07:47:07.734 [apm-remote-config-poller] ERROR co.elastic.apm.agent.report.HttpUtils - Exception when closing input stream of HttpURLConnection.
2020-01-30 07:47:07.734 [apm-remote-config-poller] ERROR co.elastic.apm.agent.configuration.ApmServerConfigurationSource - Connection refused (Connection refused)
2020-01-30 08:17:47.438 [apm-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Error sending data to APM server: Server returned HTTP response code: 503 for URL: http://apm-server-quickstart-apm-http.elast
2020-01-30 08:17:47.438 [apm-reporter] WARN co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - upstream connect error or disconnect/reset before headers. reset reason: connection failure
2020-01-30 08:17:57.443 [apm-reporter] INFO co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Backing off for 36 seconds (+/-10%)
2020-01-30 08:18:34.339 [apm-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Error sending data to APM server: Server returned HTTP response code: 503 for URL: http://apm-server-quickstart-apm-http.elast
2020-01-30 08:18:34.340 [apm-reporter] WARN co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - upstream connect error or disconnect/reset before headers. reset reason: connection failure
2020-01-30 08:18:44.346 [apm-reporter] INFO co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Backing off for 36 seconds (+/-10%)
2020-01-30 08:19:20.449 [apm-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Error sending data to APM server: Server returned HTTP response code: 503 for URL: http://apm-server-quickstart-apm-http.elast
2020-01-30 08:19:20.449 [apm-reporter] WARN co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - upstream connect error or disconnect/reset before headers. reset reason: connection failure

2

Please take a look at our troubleshooting pages- https://www.elastic.co/guide/en/apm/agent/java/current/trouble-shooting.html#trouble-shooting-no-data and https://www.elastic.co/guide/en/apm/server/master/common-problems.html

3

Maybe TLS is enabled in your APM server and you are using HTTP?

4

Yes, that was it. now disabled tls and tried to verify it worked.

Also how do i create own secret for elastic-user and apm agent authentication in eck?

5

When you use the ECK's operator to start the APM server, a secret for the secret token is created automatically. Same for the default elastic user with Kibana. Did you happen to see the blog post from two days ago?

6

Yes, but i want to provide known values for both of them, Please let me know to do that

7

Ahh, I don't know about that, but I assume it is not different from manipulating any other k8s secret (or adding new ones and using them...)

8

Regardless of secrets, you should check out this blog post, may be very useful for you for instrumenting Java app on k8s.

9

Cool, thanks

10

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.