Setup elastic-apm-agent for kafka

Elastic Observability
1

Hey guys,
I am playing around with elastic-apm-agent. The goal is to monitor kafka.
I am using the agent config given to me in Kibana:

java -javaagent:/root/kafka/elastic-apm-agent-1.28.1.jar \
-Delastic.apm.service_name=my-application \
-Delastic.apm.server_urls=http://10.0.0.41:8200 \
-Delastic.apm.secret_token= \
-Delastic.apm.environment=production \
-Delastic.apm.application_packages=org.example \
-jar my-application.jar

It seems that no matter which jar in kafka/libs folder i try in "-jar my-application.jar" line,
doesnt work.
for example if i try kafka-clients-3.0.0.jar i get this output, which is the same for any jar i try with

WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
2021-12-13 15:39:30,891 [main] INFO  co.elastic.apm.agent.util.JmxUtils - Found JVM-specific OperatingSystemMXBean interface: com.sun.management.OperatingSystemMXBean
2021-12-13 15:39:30,997 [main] INFO  co.elastic.apm.agent.configuration.StartupInfo - Starting Elastic APM 1.28.1 as my-application on Java 11.0.13 Runtime version: 11.0.13+8-LTS VM version: 11.0.13+8-LTS (Red Hat, Inc.) Linux 3.10.0-1160.49.1.el7.x86_64
2021-12-13 15:39:30,998 [main] INFO  co.elastic.apm.agent.configuration.StartupInfo - service_name: 'my-application' (source: Java System Properties)
2021-12-13 15:39:30,999 [main] INFO  co.elastic.apm.agent.configuration.StartupInfo - environment: 'production' (source: Java System Properties)
2021-12-13 15:39:30,999 [main] INFO  co.elastic.apm.agent.configuration.StartupInfo - secret_token: 'XXXX' (source: Java System Properties)
2021-12-13 15:39:30,999 [main] INFO  co.elastic.apm.agent.configuration.StartupInfo - server_urls: 'http://10.0.0.41:8200' (source: Java System Properties)
2021-12-13 15:39:31,000 [main] INFO  co.elastic.apm.agent.configuration.StartupInfo - application_packages: 'org.example' (source: Java System Properties)
2021-12-13 15:39:33,842 [main] INFO  co.elastic.apm.agent.impl.ElasticApmTracer - Tracer switched to RUNNING state
no main manifest attribute, in libs/kafka-clients-3.0.0.jar

The APM server is running and reachable via curl.
I am pretty sure i am missing the obvious, but i could use some help with this :slight_smile:

Best regards
Christian

2

Hi Christian,

Could you please describe a little further what is not working? The warning you posted is fairly generic and tells only that certain reflection operations won't be allowed, but not that the agent is not working.

Also, it is essential to highlight that the APM support for Kafka is for the applications and not for the Kafka broker. So I would presume that my-application.jar is some application using the Kafka client API to produce or consumer records.

— @riferrei

3

@riferrei
Hi Ricardo
Sorry about the latency, but Covid got the better of this household :slight_smile:
Your info on APM support being only for the applications solved my understanding issues. Thanks for that.

Best regards
Christian

1 Like