No stack trace available in JAVA APM

Hi,

I am having the latest version of Elastic Stack & Java APM agent.

Everything is working fine, but when I am open trace sample I always get No stack trace available in Span details whereas I would like to get all query details there.

Kindly assist.

This may be related to spans duration - see the span_frames_min_duration config for details.

I have added span_frames_min_duration setting, but still the same. I am getting the below warnings in tomcat catalina.out :

2020-07-29 14:01:39,841 [main] WARN co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - com.mysql.jdbc.ConnectionImpl uses an unsupported class file version (pre Java 5) and can't be instrumented. Consider updating to a newer version of that library.
2020-07-29 14:01:39,892 [main] WARN co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - com.mysql.jdbc.PreparedStatement uses an unsupported class file version (pre Java 5) and can't be instrumented. Consider updating to a newer version of that library.
2020-07-29 14:01:39,917 [main] WARN co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - com.mysql.jdbc.StatementImpl uses an unsupported class file version (pre Java 5) and can't be instrumented. Consider updating to a newer version of that library.
2020-07-29 14:01:39,941 [main] WARN co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - com.mysql.jdbc.ServerPreparedStatement uses an unsupported class file version (pre Java 5) and can't be instrumented. Consider updating to a newer version of that library.
2020-07-29 14:14:13,715 [main] WARN co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - com.thoughtworks.xstream.core.util.CompositeClassLoader uses an unsupported class file version (pre Java 5) and can't be instrumented. Consider updating to a newer version of that library.
2020-07-29 14:14:13,795 [main] WARN co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - com.thoughtworks.xstream.core.util.ClassLoaderReference uses an unsupported class file version (pre Java 5) and can't be instrumented. Consider updating to a newer version of that library.

Hold on, I am going to extend our support for Java 1.4 bytecode as well, should be quick

Can you try the relevant artifact from this snapshot build and see how it works for you?

I tried the [src/github.com/elastic/apm-agent-java/elastic-apm-agent/target/elastic-apm-agent-1.18.1.RC1-SNAPSHOT.jar] package, but still nothing. I could see the below warnings now :

2020-07-30 11:58:54,003 [main] WARN co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - com.mysql.jdbc.ConnectionImpl uses an unsupported class file version (pre Java 4}) and can't be instrumented. Consider updating to a newer version of that library.

2020-07-30 11:58:54,046 [main] WARN co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - com.mysql.jdbc.PreparedStatement uses an unsupported class file version (pre Java 4}) and can't be instrumented. Consider updating to a newer version of that library.

2020-07-30 11:58:54,068 [main] WARN co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - com.mysql.jdbc.StatementImpl uses an unsupported class file version (pre Java 4}) and can't be instrumented. Consider updating to a newer version of that library.

2020-07-30 11:58:54,088 [main] WARN co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - com.mysql.jdbc.ServerPreparedStatement uses an unsupported class file version (pre Java 4}) and can't be instrumented. Consider updating to a newer version of that library.

30-Jul-2020 12:00:23.504 WARN ING [main] org.apache.tomcat.jdbc.pool.ConnectionPool.checkPoolConfiguration maxIdle is smaller than minIdle, setting maxIdle to: 10

30-Jul-2020 12:00:46.452 WARN ING [main] org.apache.tomcat.jdbc.pool.ConnectionPool.checkPoolConfiguration maxIdle is smaller than minIdle, setting maxIdle to: 10

2020-07-30 12:01:12,514 [http-nio-8080-exec-4] WARN co.elastic.apm.agent.impl.transaction.Span - Max spans (1000) for transaction 'ProductListResource#getProductList' 00-96c6c5bb672ca66310d767f2302c314c-133189f586389a7f-01 (7812a4d2) has been reached. For this transaction and possibly others, further spans will be dropped. See config param 'transaction_max_spans'.

What do you mean? No stack traces? What did you set for span_frame_min_duration?

What mysql driver version are you using? Could it be compiled with a version lower than 1.4?

I have added below config to setenv.sh file :

export CATALINA_OPTS="$CATALINA_OPTS -javaagent:/opt/elastic-apm-agent-1.18.1.RC1-SNAPSHOT.jar"

export CATALINA_OPTS="$CATALINA_OPTS -Delastic.apm.service_name=test-apm-service"

export CATALINA_OPTS="$CATALINA_OPTS -Delastic.apm.server_urls=http://localhost:8200"

export CATALINA_OPTS="$CATALINA_OPTS -Delastic.apm.application_packages=com.nm"

export CATALINA_OPTS="$CATALINA_OPTS -Delastic.apm.stack_trace_limit=-1"

export CATALINA_OPTS="$CATALINA_OPTS -Delastic.apm.transaction_max_spans=1000"

export CATALINA_OPTS="$CATALINA_OPTS -Delastic.apm.span_frames_min_duration=-1ms"

I could not see the mysql queries for the transactions.

If your MySQL driver is compile with Java lower than 1.4 - this is expected.
If you upgrade the driver to one compiled with 1.4, you need to use the snapshot I provided above, as elastic-apm-agent-1.18.1.RC1-SNAPSHOT.jar only instruments classes compiled with 1.5 or higher.
If you upgrade the driver to one compiled with 1.5 or higher - it should work.