Apache Karaf and APM agent

Elastic Observability APM
1

Kibana version: 7.3.1

Elasticsearch version: 7.3.1

APM Server version: 7.9.3

APM Agent language and version: Java 1.18.1

Browser version: Chrome 86.0.4240.111

Original install method (e.g. download page, yum, deb, from source, etc.) and version: from source (Windows)

Fresh install or upgraded from other version? Docker compose for the stack, fresh install for APM Server

Is there anything special in your setup? No special configuration. All the stack is build with a docker-compose, except for APM.

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):

Hi everybody!
I'm trying to monitor the apache karaf container used under Talend Open Studio for ESB (v 7.2.1).
I've configured the APM server in the right way and it is reacheable by Kibana, nevertheless I've some problems configuring the APM agent, in particular, attaching it to the .jar process of the karaf.
I've tried both with the main karaf jar file (org.apache.karaf.main-4.2.4) located under /container/lib/boot/ folder and with the wrapper installed within the karaf itself, which .jar file is located under /container/lib/wrapper (karaf-wrapper).

Is there a way to configure the agent correctly?
I'm always getting the same error reported below:

1 PS C:\talend\TOS_ESB-20190620_1446-V7.2.1\Runtime_ESBSE\container\lib\wrapper> java -javaagent:C:\Users\daniele.uboldi\Downloads\jar\agents\elk\elastic-apm-agent-1.18.1.jar -Delastic.apm.service_name=Main -Delastic.apm.server_url=http://localhost:8020 -Delastic.apm.secret_token= -Delastic.apm.application_packages=org.apache.karaf.wrapper -jar karaf-wrapper-main.jar
2 2020-11-02 11:18:27,899 [main] INFO  co.elastic.apm.agent.util.JmxUtils - Found JVM-specific OperatingSystemMXBean interface: com.sun.management.OperatingSystemMXBean
3 2020-11-02 11:18:27,910 [main] INFO  co.elastic.apm.agent.configuration.StartupInfo - Starting Elastic APM 1.18.1 as karaf-wrapper-main on Java 1.8.0_251 Runtime version: 1.8.0_251-b08 VM version: 25.251-b08 (Oracle Corporation) Windows 10 10.0
4 2020-11-02 11:18:27,915 [main] WARN  co.elastic.apm.agent.configuration.StartupInfo - To enable all features and decrease startup time, please configure application_packages
5 2020-11-02 11:18:29,848 [elastic-apm-server-healthcheck] INFO  co.elastic.apm.agent.report.ApmServerHealthChecker - Elastic APM server is available: {  "build_date": "2020-10-16T08:02:43Z",  "build_sha": "af7b123f289d79c34c9da11471803f457eafa5c0",  "version": "7.9.3"}
6 2020-11-02 11:18:30,033 [main] INFO  co.elastic.apm.agent.impl.ElasticApmTracer - Tracer switched to RUNNING state
7 Errore: impossibile trovare o caricare la classe principale .apm.service_name=Main
8 2020-11-02 11:18:30,053 [elastic-apm-init-instrumentation-shutdown-hook] INFO  co.elastic.apm.agent.impl.ElasticApmTracer - Tracer switched to STOPPED state

For those who are not italian, line 7 is:

Error: unable to find or load the main class .apm.service-name=Main

Can anyone give me some help?

2

Hi @Daniele_Uboldi, welcome to our forum !

I have no experience with running Karaf, but it seems that your agent parameters are somehow truncated, as the agent do not get the value of the application_packages parameter.

Could you try to store parameters in a elasticapm.properties in the same folder as elastic-apm-agent-1.18.1.jar ? With that you will be able to remove all parameters but the -javaagent one.

3

Hi @Sylvain_Juge, thank you for your welcome and the reply!

Is there any guide on how to configure the parameters in the file?
Is it simply a configuration file like the SpringBoot ones?

Thank you in advance!

4

There are quite a few ways to configure the agent, I suggest you to have a look at our official documentation available here.

Yes, it's a similar file as the ones used to configure spring-boot, but we do not support YAML format, only the regular java properties format.

Also, please note that when using this properties files, you need to remove the elastic.apm. prefix to properties that are stored in the file. This prefix is required only for JVM command line to avoid any conflicts.

5

Thank you @Sylvain_Juge!
I've tried and now I'm getting a completely different error that is reported below:

PS C:\talend\TOS_ESB-20190620_1446-V7.2.1\Runtime_ESBSE\container\lib\boot> java -javaagent:C:\Users\daniele.uboldi\Downloads\jar\agents\elk\elastic-apm-agent-1.18.1.jar -jar org.apache.karaf.main-4.2.4.jar

2020-11-03 09:58:30,118 [main] INFO  co.elastic.apm.agent.util.JmxUtils - Found JVM-specific OperatingSystemMXBean interface: com.sun.management.OperatingSystemMXBean
2020-11-03 09:58:30,130 [main] INFO  co.elastic.apm.agent.configuration.StartupInfo - Starting Elastic APM 1.18.1 as karaf-monitoring on Java 1.8.0_251 Runtime version: 1.8.0_251-b08 VM version: 25.251-b08 (Oracle Corporation) Windows 10 10.0
2020-11-03 09:58:32,241 [main] INFO  co.elastic.apm.agent.impl.ElasticApmTracer - Tracer switched to RUNNING state
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: org/osgi/framework/BundleException
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Unknown Source)
        at java.lang.Class.privateGetMethodRecursive(Unknown Source)
        at java.lang.Class.getMethod0(Unknown Source)
        at java.lang.Class.getMethod(Unknown Source)
        at sun.launcher.LauncherHelper.validateMainClass(Unknown Source)
        at sun.launcher.LauncherHelper.checkAndLoadMain(Unknown Source)
Caused by: java.lang.ClassNotFoundException: org.osgi.framework.BundleException
        at java.net.URLClassLoader.findClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        ... 7 more
2020-11-03 09:58:32,268 [elastic-apm-init-instrumentation-shutdown-hook] INFO  co.elastic.apm.agent.impl.ElasticApmTracer - Tracer switched to STOPPED state

I was also thinking about your previous answer, is it possible that the agent does not get the application.packages parameters from the command line because the previous parameter (agent.token) is empty?

6

This second error message is related to OSGI, which is different from regular applications classloading.

When looking at past forum posts & agent issues, I've found this : https://github.com/elastic/apm-agent-java/issues/436.

Could you try to set the boot_delegation_packages configuration option as described in the issue ?

7

I've added the boot_delegation_packages to the elasticapm.properties file, as mentioned in one of the last comments by eyalkoren, but with no results, the error is still appearing to me.
My elasticapm.properties file look like this now:

service_name=karaf-monitoring
application_packages=org.apache.karaf
server_urls=http://localhost:8020
boot_delegation_packages=org.apache.karaf.jaas.boot,org.apache.karaf.jaas.boot.principal,org.apache.karaf.management.boot,sun.*,com.sun.*,javax.transaction,javax.transaction.*,javax.xml.crypto,javax.xml.crypto.*,org.apache.xerces.jaxp.datatype,org.apache.xerces.stax,org.apache.xerces.parsers,org.apache.xerces.jaxp,org.apache.xerces.jaxp.validation,org.apache.xerces.dom,co.elastic.apm.agent.*

Do you have any other options to try?

8

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.