Apm-agent-java not able to transmit logs via `log_sending` to apm-server with delayed start

niecore · March 7, 2025, 7:42am

Kibana version: 8.17.2

Elasticsearch version: 8.17.2

APM Server version: 8.17.2

APM Agent language and version: apm-agent-java:1.52.0

Browser version: not relevant

Original install method (e.g. download page, yum, deb, from source, etc.) and version: elk stack in docker-compose and apm-agent from docker.elastic.co/observability/apm-agent-java:1.52.0

Fresh install or upgraded from other version?: Fresh install

Is there anything special in your setup?: Starting apm-server with 10 second delay after the java application.

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):

Dear Elasticsearch-Team,

I observed following issue while transmitting logs from apm-agent-java to an apm-server via the experimental log_sending. When the java application is started before the apm-server, the log transmission is not working because the apm-agent-java did not receive the apm-server version (see last log line in error log below). Traces and metrics are working as expected. Changing the values of ELASTIC_APM_SERVER_TIMEOUT or ELASTIC_APM_API_REQUEST_TIME did not solve the problem.

Steps to reproduce:

start elasticsearch and kibana
start java application with apm-agent-java
wait 10 seconds
start apm-server

Provide logs and/or server output (if relevant):

[java-app]   | 2025-03-06 12:37:38,954 [main] INFO  co.elastic.apm.agent.util.JmxUtils - Found JVM-specific OperatingSystemMXBean interface: com.sun.management.OperatingSystemMXBean
[java-app]   | 2025-03-06 12:37:38,973 [main] INFO  co.elastic.apm.agent.util.JmxUtils - Found JVM-specific ThreadMXBean interface: com.sun.management.ThreadMXBean
[java-app]   | 2025-03-06 12:37:38,979 [main] INFO  co.elastic.apm.agent.configuration.StartupInfo - Starting Elastic APM 1.52.0 as java-demo-app on Java 17.0.14 Runtime version: 17.0.14+7 VM version: 17.0.14+7 (Eclipse Adoptium) Linux 6.12.7-200.fc41.aarch64
[java-app]   | 2025-03-06 12:37:38,980 [main] INFO  co.elastic.apm.agent.configuration.StartupInfo - service_name: 'java-demo-app' (source: Environment Variables)
[java-app]   | 2025-03-06 12:37:38,980 [main] INFO  co.elastic.apm.agent.configuration.StartupInfo - environment: 'development' (source: Environment Variables)
[java-app]   | 2025-03-06 12:37:38,980 [main] INFO  co.elastic.apm.agent.configuration.StartupInfo - server_url: 'http://apm-server:8200' (source: Environment Variables)
[java-app]   | 2025-03-06 12:37:38,980 [main] INFO  co.elastic.apm.agent.configuration.StartupInfo - application_packages: 'com.example' (source: Environment Variables)
[java-app]   | 2025-03-06 12:37:38,980 [main] INFO  co.elastic.apm.agent.configuration.StartupInfo - log_sending: 'true' (source: Environment Variables)
[java-app]   | 2025-03-06 12:37:39,503 [main] INFO  co.elastic.apm.agent.impl.ElasticApmTracer - Tracer switched to RUNNING state
[java-app]   | 2025-03-06 12:37:39,508 [elastic-apm-server-healthcheck] WARN  co.elastic.apm.agent.report.ApmServerHealthChecker - Elastic APM server http://apm-server:8200/ is not available (Connection refused)
[java-app]   | 2025-03-06 12:37:39,509 [elastic-apm-server-reporter] ERROR co.elastic.apm.agent.report.AbstractIntakeApiHandler - Error trying to connect to APM Server at http://apm-server:8200/intake/v2/events. Although not necessarily related to SSL, some related SSL configurations corresponding the current connection are logged at INFO level.
[java-app]   | 2025-03-06 12:37:39,510 [elastic-apm-server-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Failed to handle event of type BYTES_LOG with this error: Connection refused
[java-app]   | 2025-03-06 12:37:39,511 [elastic-apm-remote-config-poller] ERROR co.elastic.apm.agent.configuration.ApmServerConfigurationSource - Connection refused
[java-app]   | 2025-03-06 12:37:39,511 [elastic-apm-server-reporter] INFO  co.elastic.apm.agent.report.AbstractIntakeApiHandler - Backing off for 0 seconds (+/-10%)
[java-app]   | 2025-03-06 12:37:39,513 [elastic-apm-server-reporter] ERROR co.elastic.apm.agent.report.AbstractIntakeApiHandler - Error trying to connect to APM Server at http://apm-server:8200/intake/v2/events. Although not necessarily related to SSL, some related SSL configurations corresponding the current connection are logged at INFO level.
[java-app]   | 2025-03-06 12:37:39,513 [elastic-apm-server-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Failed to handle event of type BYTES_LOG with this error: Connection refused
[java-app]   | 2025-03-06 12:37:39,513 [elastic-apm-server-reporter] INFO  co.elastic.apm.agent.report.AbstractIntakeApiHandler - Backing off for 1 seconds (+/-10%)
[java-app]   | 12:37:39.569 [Timer-0] INFO com.example.DemoApp -- Sample application heartbeat - 1741264659568
[java-app]   | 2025-03-06 12:37:40,432 [elastic-apm-server-reporter] ERROR co.elastic.apm.agent.report.AbstractIntakeApiHandler - Error trying to connect to APM Server at http://apm-server:8200/intake/v2/events. Although not necessarily related to SSL, some related SSL configurations corresponding the current connection are logged at INFO level.
[java-app]   | 2025-03-06 12:37:40,432 [elastic-apm-server-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Failed to handle event of type BYTES_LOG with this error: Connection refused
[java-app]   | 2025-03-06 12:37:40,432 [elastic-apm-server-reporter] INFO  co.elastic.apm.agent.report.AbstractIntakeApiHandler - Backing off for 4 seconds (+/-10%)
[java-app]   | 12:37:40.557 [Timer-0] INFO com.example.DemoApp -- Sample application heartbeat - 1741264660557
[java-app]   | 12:37:41.553 [Timer-0] INFO com.example.DemoApp -- Sample application heartbeat - 1741264661553
[java-app]   | 12:37:42.553 [Timer-0] INFO com.example.DemoApp -- Sample application heartbeat - 1741264662553
[java-app]   | 12:37:43.553 [Timer-0] INFO com.example.DemoApp -- Sample application heartbeat - 1741264663553
[java-app]   | 2025-03-06 12:37:44,196 [elastic-apm-server-reporter] ERROR co.elastic.apm.agent.report.AbstractIntakeApiHandler - Error trying to connect to APM Server at http://apm-server:8200/intake/v2/events. Although not necessarily related to SSL, some related SSL configurations corresponding the current connection are logged at INFO level.
[java-app]   | 2025-03-06 12:37:44,196 [elastic-apm-server-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Failed to handle event of type BYTES_LOG with this error: Connection refused
[java-app]   | 2025-03-06 12:37:44,197 [elastic-apm-server-reporter] INFO  co.elastic.apm.agent.report.AbstractIntakeApiHandler - Backing off for 9 seconds (+/-10%)
[java-app]   | 12:37:44.555 [Timer-0] INFO com.example.DemoApp -- Sample application heartbeat - 1741264664555
[java-app]   | 12:37:45.556 [Timer-0] INFO com.example.DemoApp -- Sample application heartbeat - 1741264665556
[java-app]   | 12:37:46.554 [Timer-0] INFO com.example.DemoApp -- Sample application heartbeat - 1741264666554
[java-app]   | 12:37:47.552 [Timer-0] INFO com.example.DemoApp -- Sample application heartbeat - 1741264667552
[java-app]   | 12:37:48.553 [Timer-0] INFO com.example.DemoApp -- Sample application heartbeat - 1741264668553
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.671Z","log.origin":{"function":"github.com/elastic/apm-server/internal/beatcmd.(*Beat).init","file.name":"beatcmd/beat.go","file.line":146},"message":"Home path: [/usr/share/apm-server] Config path: [/usr/share/apm-server] Data path: [/usr/share/apm-server/data] Logs path: [/usr/share/apm-server/logs]","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.673Z","log.origin":{"function":"github.com/elastic/apm-server/internal/beatcmd.(*Beat).init","file.name":"beatcmd/beat.go","file.line":153},"message":"Beat ID: 20331cd8-4a0f-4ea1-a4f7-af10923332f4","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.674Z","log.logger":"beat","log.origin":{"function":"github.com/elastic/apm-server/internal/beatcmd.logSystemInfo","file.name":"beatcmd/beat.go","file.line":577},"message":"Beat info","service.name":"apm-server","system_info":{"beat":{"path":{"config":"/usr/share/apm-server","data":"/usr/share/apm-server/data","home":"/usr/share/apm-server","logs":"/usr/share/apm-server/logs"},"type":"apm-server","uuid":"20331cd8-4a0f-4ea1-a4f7-af10923332f4"},"ecs.version":"1.6.0"}}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.674Z","log.logger":"beat","log.origin":{"function":"github.com/elastic/apm-server/internal/beatcmd.logSystemInfo","file.name":"beatcmd/beat.go","file.line":585},"message":"Build info","service.name":"apm-server","system_info":{"build":{"commit":"f5450d68eca8ef6139dcbfe95a3ff101c7eb830f","time":"2025-02-05T14:40:32.000Z","version":"8.17.2"},"ecs.version":"1.6.0"}}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.674Z","log.logger":"beat","log.origin":{"function":"github.com/elastic/apm-server/internal/beatcmd.logSystemInfo","file.name":"beatcmd/beat.go","file.line":588},"message":"Go runtime info","service.name":"apm-server","system_info":{"go":{"os":"linux","arch":"arm64","max_procs":6,"version":"go1.23.5"},"ecs.version":"1.6.0"}}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.675Z","log.logger":"beat","log.origin":{"function":"github.com/elastic/apm-server/internal/beatcmd.logSystemInfo","file.name":"beatcmd/beat.go","file.line":592},"message":"Host info","service.name":"apm-server","system_info":{"host":{"architecture":"aarch64","native_architecture":"aarch64","boot_time":"2025-03-06T10:14:39Z","containerized":false,"name":"a75dfcf15065","ip":["127.0.0.1/8","10.89.1.11/24","::1/128","fe80::7052:8dff:feb5:9780/64"],"kernel_version":"6.12.7-200.fc41.aarch64","mac":["72:52:8d:b5:97:80"],"os":{"type":"linux","family":"debian","platform":"ubuntu","name":"Ubuntu","version":"22.04.5 LTS (Jammy Jellyfish)","major":22,"minor":4,"patch":5,"codename":"jammy"},"timezone":"UTC","timezone_offset_sec":0},"ecs.version":"1.6.0"}}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.676Z","log.origin":{"function":"github.com/elastic/apm-server/internal/beatcmd.adjustMaxProcs.diffInfof.func2","file.name":"beatcmd/maxprocs.go","file.line":68},"message":"maxprocs: Leaving GOMAXPROCS=6: CPU quota undefined","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.676Z","log.logger":"beat","log.origin":{"function":"github.com/elastic/apm-server/internal/beatcmd.logSystemInfo","file.name":"beatcmd/beat.go","file.line":621},"message":"Process info","service.name":"apm-server","system_info":{"process":{"capabilities":{"inheritable":null,"permitted":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","sys_chroot","setfcap"],"effective":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","sys_chroot","setfcap"],"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","sys_chroot","setfcap"],"ambient":null},"cwd":"/usr/share/apm-server","exe":"/usr/share/apm-server/apm-server","name":"apm-server","pid":2,"ppid":1,"seccomp":{"mode":"filter","no_new_privs":false},"start_time":"2025-03-06T12:37:38.270Z"},"ecs.version":"1.6.0"}}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.677Z","log.logger":"config","log.origin":{"function":"github.com/elastic/apm-server/internal/beater/config.(*AgentConfig).setup","file.name":"config/agentconfig.go","file.line":70},"message":"using output.elasticsearch for fetching agent config","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.677Z","log.logger":"config","log.origin":{"function":"github.com/elastic/apm-server/internal/beater/config.(*RumConfig).setup","file.name":"config/rum.go","file.line":84},"message":"Using default sourcemap Elasticsearch config","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.678Z","log.logger":"beater","log.origin":{"function":"github.com/elastic/apm-server/internal/beater.listen","file.name":"beater/http.go","file.line":142},"message":"Listening on: [::]:8200","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.678Z","log.origin":{"function":"github.com/elastic/apm-server/internal/beatcmd.(*Beat).Run","file.name":"beatcmd/beat.go","file.line":395},"message":"apm-server started.","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.681Z","log.logger":"beater","log.origin":{"function":"github.com/elastic/apm-server/internal/beater.processMemoryLimit","file.name":"beater/beater.go","file.line":1031},"message":"no cgroups detected, falling back to total system memory","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.681Z","log.logger":"beater","log.origin":{"function":"github.com/elastic/apm-server/internal/beater.(*Runner).Run","file.name":"beater/beater.go","file.line":191},"message":"MaxConcurrentDecoders set to 996 based on 80 percent of 9.7gb of memory","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.681Z","log.logger":"beater","log.origin":{"function":"github.com/elastic/apm-server/internal/beater.(*Runner).Run","file.name":"beater/beater.go","file.line":198},"message":"Aggregation.MaxServices set to 9733 based on 9.7gb of memory","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.681Z","log.logger":"beater","log.origin":{"function":"github.com/elastic/apm-server/internal/beater.(*Runner).Run","file.name":"beater/beater.go","file.line":205},"message":"Aggregation.ServiceTransactions.MaxGroups for service aggregation set to 9733 based on 9.7gb of memory","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.681Z","log.logger":"beater","log.origin":{"function":"github.com/elastic/apm-server/internal/beater.(*Runner).Run","file.name":"beater/beater.go","file.line":212},"message":"Aggregation.Transactions.MaxGroups set to 48667 based on 9.7gb of memory","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.681Z","log.logger":"beater","log.origin":{"function":"github.com/elastic/apm-server/internal/beater.(*Runner).Run","file.name":"beater/beater.go","file.line":219},"message":"Aggregation.ServiceDestinations.MaxGroups set to 53667 based on 9.7gb of memory","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.684Z","log.logger":"beater","log.origin":{"function":"github.com/elastic/apm-server/internal/beater.waitReady","file.name":"beater/waitready.go","file.line":40},"message":"blocking ingestion until all preconditions are satisfied","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.684Z","log.logger":"beater","log.origin":{"function":"github.com/elastic/apm-server/internal/beater.docappenderConfig","file.name":"beater/beater.go","file.line":833},"message":"docappender.DocumentBufferSize set to 7973 based on 9.7gb of memory","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.684Z","log.logger":"beater","log.origin":{"function":"github.com/elastic/apm-server/internal/beater.docappenderConfig","file.name":"beater/beater.go","file.line":850},"message":"docappender.MaxRequests set to 24 based on 9.7gb of memory","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.686Z","log.logger":"handler","log.origin":{"function":"github.com/elastic/apm-server/internal/beater/api.NewMux","file.name":"api/mux.go","file.line":138},"message":"Path / added to request handler","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.686Z","log.logger":"handler","log.origin":{"function":"github.com/elastic/apm-server/internal/beater/api.NewMux","file.name":"api/mux.go","file.line":138},"message":"Path /config/v1/agents added to request handler","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.687Z","log.logger":"handler","log.origin":{"function":"github.com/elastic/apm-server/internal/beater/api.NewMux","file.name":"api/mux.go","file.line":138},"message":"Path /config/v1/rum/agents added to request handler","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.687Z","log.logger":"handler","log.origin":{"function":"github.com/elastic/apm-server/internal/beater/api.NewMux","file.name":"api/mux.go","file.line":138},"message":"Path /intake/v2/rum/events added to request handler","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.687Z","log.logger":"handler","log.origin":{"function":"github.com/elastic/apm-server/internal/beater/api.NewMux","file.name":"api/mux.go","file.line":138},"message":"Path /intake/v3/rum/events added to request handler","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.687Z","log.logger":"handler","log.origin":{"function":"github.com/elastic/apm-server/internal/beater/api.NewMux","file.name":"api/mux.go","file.line":138},"message":"Path /intake/v2/events added to request handler","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.687Z","log.logger":"handler","log.origin":{"function":"github.com/elastic/apm-server/internal/beater/api.NewMux","file.name":"api/mux.go","file.line":138},"message":"Path /v1/traces added to request handler","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.687Z","log.logger":"handler","log.origin":{"function":"github.com/elastic/apm-server/internal/beater/api.NewMux","file.name":"api/mux.go","file.line":138},"message":"Path /v1/metrics added to request handler","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.687Z","log.logger":"handler","log.origin":{"function":"github.com/elastic/apm-server/internal/beater/api.NewMux","file.name":"api/mux.go","file.line":138},"message":"Path /v1/logs added to request handler","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.687Z","log.origin":{"function":"github.com/elastic/apm-server/internal/beater/jaeger.RegisterGRPCServices","file.name":"jaeger/grpc.go","file.line":80},"message":"deprecation notice: support for Jaeger will be removed in an upcoming version","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.687Z","log.logger":"beater","log.origin":{"function":"github.com/elastic/apm-server/internal/beater.server.run","file.name":"beater/server.go","file.line":215},"message":"Starting apm-server [f5450d68eca8ef6139dcbfe95a3ff101c7eb830f built 2025-02-05 14:40:32 +0000 UTC]. Hit CTRL-C to stop it.","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.687Z","log.logger":"beater","log.origin":{"function":"github.com/elastic/apm-server/internal/beater.(*httpServer).start","file.name":"beater/http.go","file.line":86},"message":"RUM endpoints enabled!","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"warn","@timestamp":"2025-03-06T12:37:48.687Z","log.logger":"beater","log.origin":{"function":"github.com/elastic/apm-server/internal/beater.(*httpServer).start","file.name":"beater/http.go","file.line":89},"message":"CORS related setting `apm-server.rum.allow_origins` allows all origins. Consider more restrictive setting for production use.","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.687Z","log.logger":"beater","log.origin":{"function":"github.com/elastic/apm-server/internal/beater.(*httpServer).start","file.name":"beater/http.go","file.line":104},"message":"SSL disabled.","service.name":"apm-server","ecs.version":"1.6.0"}
[apm-server] | {"log.level":"info","@timestamp":"2025-03-06T12:37:48.691Z","log.logger":"beater","log.origin":{"function":"github.com/elastic/apm-server/internal/beater.waitReady","file.name":"beater/waitready.go","file.line":68},"message":"no longer blocking ingestion as all precondition checks are now satisfied","service.name":"apm-server","ecs.version":"1.6.0"}
[java-app]   | 12:37:49.556 [Timer-0] INFO com.example.DemoApp -- Sample application heartbeat - 1741264669556
[java-app]   | 12:37:50.552 [Timer-0] INFO com.example.DemoApp -- Sample application heartbeat - 1741264670552
[java-app]   | 12:37:51.556 [Timer-0] INFO com.example.DemoApp -- Sample application heartbeat - 1741264671556
[java-app]   | 12:37:52.553 [Timer-0] INFO com.example.DemoApp -- Sample application heartbeat - 1741264672553
[java-app]   | 2025-03-06 12:37:52,940 [elastic-apm-server-reporter] WARN  co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - sending logs to apm server is not supported, upgrading to a more recent version is required

Sylvain_Juge · March 7, 2025, 8:02am

Hi @niecore ,

When the apm agent starts, there is a call to a "healthcheck" API endpoint that provides the version of APM server. This version is then used by the agent to select which endpoints are available and how the data should be serialized in a specific version.

When the server is not available, the agent has to assume a conservative option where a very old (something like 6.x or 7.x) backend might be used.

This behavior can't be changed through configuration unfortunately. Another problem with that is that the agent will likely produce lots of error messages in the logs when the server is not available.

One possible alternative would be to use an OpenTelemetry agent or our EDOT agent as those does not require such API call on agent start, the backend has to deal on its own with any variation in the produced data.

niecore · March 7, 2025, 8:30am

@Sylvain_Juge Thanks for the super fast help

Would it also be reasonable to use an EDOT Collector for container log collection? Do I need to disable log transmission in the EDOT Agent then?

Jonas_Kunz · March 7, 2025, 11:50am

Using the collector to collect logs directly from containers is actually the preferred option here: When the EDOT Agent sends logs, it might not be able to send all the logs if the JVM crashes. If you collect them as container logs via the collector instead, logs will be collected fully, even if the container terminates abnormally.

Topic		Replies	Views
APM Java Agent APM docker , java	3	573	January 5, 2021
Beginer error, sent data to APM but this is not showed on kibana APM java	10	1650	January 26, 2022
APM agent does not start sending any events APM java	2	164	April 19, 2024
APM Server status is You have correctly setup APM Serve But Agent status is No data has been received APM docker , java	2	251	October 25, 2023
Traces are not shown in Kibana APM docker , java , server	31	4031	March 20, 2022

Apm-agent-java not able to transmit logs via `log_sending` to apm-server with delayed start

Related topics