ERROR co.elastic.apm.agent.bci.IndyBootstrap - Advice threw an exception, this should never happen! java.lang.NoSuchMethodError

Kibana version: 8.6.2

Elasticsearch version: 8.6.2

APM Server version: 8.6.2

APM Agent language and version: java/1.43.0

Original install method (e.g. download page, yum, deb, from source, etc.) and version: Docker stack from registry

Fresh install or upgraded from other version? Fresh install

Is there anything special in your setup? Nothing in specific

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):

Hello.

We're running a Springboot application, and we get an error in logs on startup.

Apparently, all transactions are showing in kibana. But our developers are question me about the error and I can't give us a correct answer.

The log below is a part of error:

2023-11-27 09:46:16.134  INFO 7 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring FrameworkServlet 'dispatcherServlet'
2023-11-27 09:46:19.372  INFO 7 --- [nio-8080-exec-1] io.lettuce.core.EpollProvider            : Starting without optional epoll library
2023-11-27 09:46:19.432  INFO 7 --- [nio-8080-exec-1] io.lettuce.core.KqueueProvider           : Starting without optional kqueue library
2023-11-27 09:46:20,268 [http-nio-8080-exec-1] ERROR co.elastic.apm.agent.bci.IndyBootstrap - Advice threw an exception, this should never happen!
java.lang.NoSuchMethodError: reactor.core.publisher.Operators.liftPublisher(Ljava/util/function/BiFunction;)Ljava/util/function/Function;
  at co.elastic.apm.agent.reactor.TracedSubscriber.wrapOperators(TracedSubscriber.java:243) ~[?:?]
  at co.elastic.apm.agent.reactor.TracedSubscriber.registerHooks(TracedSubscriber.java:221) ~[?:?]
  at co.elastic.apm.agent.reactor.ReactorInstrumentation$RegisterHookAdvice.onEnter(ReactorInstrumentation.java:72) ~[elastic-apm-agent.jar:1.43.0]
  at reactor.core.publisher.Mono.onAssembly(Mono.java:3656) ~[reactor-core-3.1.5.RELEASE.jar!/:3.1.5.RELEASE]
  at reactor.core.publisher.Mono.defer(Mono.java:189) ~[reactor-core-3.1.5.RELEASE.jar!/:3.1.5.RELEASE]
  at org.springframework.data.redis.connection.lettuce.LettuceReactiveRedisConnection$AsyncConnect.<init>(LettuceReactiveRedisConnection.java:339) ~[spring-data-redis-2.0.5.RELEASE.jar!/:2.0.5.RELEASE]
  at org.springframework.data.redis.connection.lettuce.LettuceReactiveRedisConnection.<init>(LettuceReactiveRedisConnection.java:84) ~[spring-data-redis-2.0.5.RELEASE.jar!/:2.0.5.RELEASE]
  at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.getReactiveConnection(LettuceConnectionFactory.java:329) ~[spring-data-redis-2.0.5.RELEASE.jar!/:2.0.5.RELEASE]
  at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.getReactiveConnection(LettuceConnectionFactory.java:81) ~[spring-data-redis-2.0.5.RELEASE.jar!/:2.0.5.RELEASE]
  at org.springframework.boot.actuate.redis.RedisReactiveHealthIndicator.doHealthCheck(RedisReactiveHealthIndicator.java:48) ~[spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
  at org.springframework.boot.actuate.health.AbstractReactiveHealthIndicator.health(AbstractReactiveHealthIndicator.java:34) ~[spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
  at org.springframework.boot.actuate.autoconfigure.health.HealthIndicatorBeansComposite$ReactiveHealthIndicators.lambda$adapt$1(HealthIndicatorBeansComposite.java:75) ~[spring-boot-actuator-autoconfigure-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
  at org.springframework.boot.actuate.health.CompositeHealthIndicator.health(CompositeHealthIndicator.java:68) ~[spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
  at org.springframework.boot.actuate.health.HealthEndpointWebExtension.getHealth(HealthEndpointWebExtension.java:50) ~[spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_372]
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_372]
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_372]
  at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_372]
  at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:223) ~[spring-core-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
  at org.springframework.boot.actuate.endpoint.invoke.reflect.ReflectiveOperationInvoker.invoke(ReflectiveOperationInvoker.java:76) ~[spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
  at org.springframework.boot.actuate.endpoint.annotation.AbstractDiscoveredOperation.invoke(AbstractDiscoveredOperation.java:61) ~[spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
  at org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$ServletWebOperationAdapter.handle(AbstractWebMvcEndpointHandlerMapping.java:243) [spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
  at org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$OperationHandler.handle(AbstractWebMvcEndpointHandlerMapping.java:299) [spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_372]
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_372]
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_372]
  at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_372]
  at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:209) [spring-web-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
  at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136) [spring-web-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
  at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102) [spring-webmvc-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
  at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:870) [spring-webmvc-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
  at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:776) [spring-webmvc-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
  at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) [spring-webmvc-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
  at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:991) [spring-webmvc-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
  at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:925) [spring-webmvc-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
  at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:978) [spring-webmvc-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
  at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:870) [spring-webmvc-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:635) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:855) [spring-webmvc-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) [tomcat-embed-websocket-8.5.28.jar!/:8.5.28]
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:158) [spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
  at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:126) [spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
  at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:111) [spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
  at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.springframework.boot.actuate.web.trace.servlet.HttpTraceFilter.doFilterInternal(HttpTraceFilter.java:84) [spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
  at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) [spring-web-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
  at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:496) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_372]
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_372]
  at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
  at java.lang.Thread.run(Thread.java:750) [?:1.8.0_372]

And the log below is with debug level:

2023-11-27 16:41:11,506 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation OnComplete: hasSuperType(erasure(name(equals(io.lettuce.core.protocol.RedisCommand)))) matches interface io.lettuce.core.protocol.RedisCommand
2023-11-27 16:41:11,507 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation OnCompleteExceptionally: hasSuperType(erasure(name(equals(io.lettuce.core.protocol.RedisCommand)))) matches interface io.lettuce.core.protocol.RedisCommand
2023-11-27 16:41:11,507 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation OnCancel: hasSuperType(erasure(name(equals(io.lettuce.core.protocol.RedisCommand)))) matches interface io.lettuce.core.protocol.RedisCommand
2023-11-27 16:41:11,894 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation OnComplete: hasSuperType(erasure(name(equals(io.lettuce.core.protocol.RedisCommand)))) matches class io.lettuce.core.protocol.AsyncCommand
2023-11-27 16:41:11,894 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation OnCompleteExceptionally: hasSuperType(erasure(name(equals(io.lettuce.core.protocol.RedisCommand)))) matches class io.lettuce.core.protocol.AsyncCommand
2023-11-27 16:41:11,895 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation OnCancel: hasSuperType(erasure(name(equals(io.lettuce.core.protocol.RedisCommand)))) matches class io.lettuce.core.protocol.AsyncCommand
2023-11-27 16:41:11,916 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation OnComplete: (name(equals(complete)) and hasParameter(ofSize(0)) and not(isAbstract())) matches public void io.lettuce.core.protocol.AsyncCommand.complete()
2023-11-27 16:41:11,923 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation OnCompleteExceptionally: (name(equals(completeExceptionally)) and hasParameter(hasTypes(erasures(containing(is(class java.lang.Throwable))))) and not(isAbstract())) matches public boolean io.lettuce.core.protocol.AsyncCommand.completeExceptionally(java.lang.Throwable)
2023-11-27 16:41:11,924 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation OnCancel: (name(equals(cancel)) and hasParameter(ofSize(0)) and not(isAbstract())) matches public void io.lettuce.core.protocol.AsyncCommand.cancel()
2023-11-27 16:41:12.391  INFO 7 --- [nio-8080-exec-2] io.lettuce.core.EpollProvider            : Starting without optional epoll library
2023-11-27 16:41:12.396  INFO 7 --- [nio-8080-exec-2] io.lettuce.core.KqueueProvider           : Starting without optional kqueue library
2023-11-27 16:41:14,115 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.IndyPluginClassLoaderFactory - Creating plugin class loader for org.springframework.boot.loader.LaunchedURLClassLoader@579bb367 containing [co.elastic.apm.agent.reactor.TracedSubscriber$1, co.elastic.apm.agent.reactor.TracedSubscriber, co.elastic.apm.agent.reactor.ReactorInstrumentation$RegisterHookAdvice, co.elastic.apm.agent.reactor.package-info, co.elastic.apm.agent.reactor.ReactorInstrumentation, co.elastic.apm.agent.bci.classloading.LookupExposer]
2023-11-27 16:41:14,790 [http-nio-8080-exec-3] DEBUG co.elastic.apm.agent.httpserver.HttpServerHelper - Not tracing this request as the path /protocolo/metrics is ignored by the matcher */metrics
2023-11-27 16:41:14,783 [http-nio-8080-exec-2] ERROR co.elastic.apm.agent.bci.IndyBootstrap - Advice threw an exception, this should never happen!
java.lang.NoSuchMethodError: reactor.core.publisher.Operators.liftPublisher(Ljava/util/function/BiFunction;)Ljava/util/function/Function;
	at co.elastic.apm.agent.reactor.TracedSubscriber.wrapOperators(TracedSubscriber.java:243) ~[?:?]
	at co.elastic.apm.agent.reactor.TracedSubscriber.registerHooks(TracedSubscriber.java:221) ~[?:?]
	at co.elastic.apm.agent.reactor.ReactorInstrumentation$RegisterHookAdvice.onEnter(ReactorInstrumentation.java:72) ~[elastic-apm-agent.jar:1.43.0]
	at reactor.core.publisher.Mono.onAssembly(Mono.java:3656) ~[reactor-core-3.1.5.RELEASE.jar!/:3.1.5.RELEASE]
	at reactor.core.publisher.Mono.defer(Mono.java:189) ~[reactor-core-3.1.5.RELEASE.jar!/:3.1.5.RELEASE]
	at org.springframework.data.redis.connection.lettuce.LettuceReactiveRedisConnection$AsyncConnect.<init>(LettuceReactiveRedisConnection.java:339) ~[spring-data-redis-2.0.5.RELEASE.jar!/:2.0.5.RELEASE]
	at org.springframework.data.redis.connection.lettuce.LettuceReactiveRedisConnection.<init>(LettuceReactiveRedisConnection.java:84) ~[spring-data-redis-2.0.5.RELEASE.jar!/:2.0.5.RELEASE]
	at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.getReactiveConnection(LettuceConnectionFactory.java:329) ~[spring-data-redis-2.0.5.RELEASE.jar!/:2.0.5.RELEASE]
	at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.getReactiveConnection(LettuceConnectionFactory.java:81) ~[spring-data-redis-2.0.5.RELEASE.jar!/:2.0.5.RELEASE]
	at org.springframework.boot.actuate.redis.RedisReactiveHealthIndicator.doHealthCheck(RedisReactiveHealthIndicator.java:48) ~[spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
	at org.springframework.boot.actuate.health.AbstractReactiveHealthIndicator.health(AbstractReactiveHealthIndicator.java:34) ~[spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
	at org.springframework.boot.actuate.autoconfigure.health.HealthIndicatorBeansComposite$ReactiveHealthIndicators.lambda$adapt$1(HealthIndicatorBeansComposite.java:75) ~[spring-boot-actuator-autoconfigure-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
	at org.springframework.boot.actuate.health.CompositeHealthIndicator.health(CompositeHealthIndicator.java:68) ~[spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
	at org.springframework.boot.actuate.health.HealthEndpointWebExtension.getHealth(HealthEndpointWebExtension.java:50) ~[spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_372]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_372]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_372]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_372]
	at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:223) ~[spring-core-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
	at org.springframework.boot.actuate.endpoint.invoke.reflect.ReflectiveOperationInvoker.invoke(ReflectiveOperationInvoker.java:76) ~[spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
	at org.springframework.boot.actuate.endpoint.annotation.AbstractDiscoveredOperation.invoke(AbstractDiscoveredOperation.java:61) ~[spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
	at org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$ServletWebOperationAdapter.handle(AbstractWebMvcEndpointHandlerMapping.java:243) [spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
	at org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$OperationHandler.handle(AbstractWebMvcEndpointHandlerMapping.java:299) [spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_372]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_372]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_372]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_372]
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:209) [spring-web-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136) [spring-web-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102) [spring-webmvc-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:870) [spring-webmvc-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:776) [spring-webmvc-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) [spring-webmvc-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:991) [spring-webmvc-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:925) [spring-webmvc-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:978) [spring-webmvc-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:870) [spring-webmvc-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:635) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:855) [spring-webmvc-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) [tomcat-embed-websocket-8.5.28.jar!/:8.5.28]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:158) [spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:126) [spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:111) [spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.springframework.boot.actuate.web.trace.servlet.HttpTraceFilter.doFilterInternal(HttpTraceFilter.java:84) [spring-boot-actuator-2.0.0.RELEASE.jar!/:2.0.0.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) [spring-web-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:496) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_372]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_372]
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.28.jar!/:8.5.28]
	at java.lang.Thread.run(Thread.java:750) [?:1.8.0_372]
2023-11-27 16:41:15,188 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation OnComplete: hasSuperType(erasure(name(equals(io.lettuce.core.protocol.RedisCommand)))) matches class io.lettuce.core.protocol.Command
2023-11-27 16:41:15,188 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation OnCompleteExceptionally: hasSuperType(erasure(name(equals(io.lettuce.core.protocol.RedisCommand)))) matches class io.lettuce.core.protocol.Command
2023-11-27 16:41:15,188 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation OnCancel: hasSuperType(erasure(name(equals(io.lettuce.core.protocol.RedisCommand)))) matches class io.lettuce.core.protocol.Command
2023-11-27 16:41:15,197 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation OnCompleteExceptionally: (name(equals(completeExceptionally)) and hasParameter(hasTypes(erasures(containing(is(class java.lang.Throwable))))) and not(isAbstract())) matches public boolean io.lettuce.core.protocol.Command.completeExceptionally(java.lang.Throwable)
2023-11-27 16:41:15,199 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation OnComplete: (name(equals(complete)) and hasParameter(ofSize(0)) and not(isAbstract())) matches public void io.lettuce.core.protocol.Command.complete()
2023-11-27 16:41:15,200 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation OnCancel: (name(equals(cancel)) and hasParameter(ofSize(0)) and not(isAbstract())) matches public void io.lettuce.core.protocol.Command.cancel()
2023-11-27 16:41:15,217 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation OnComplete: hasSuperType(erasure(name(equals(io.lettuce.core.protocol.RedisCommand)))) matches class io.lettuce.core.RedisPublisher$SubscriptionCommand
2023-11-27 16:41:15,218 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation OnCompleteExceptionally: hasSuperType(erasure(name(equals(io.lettuce.core.protocol.RedisCommand)))) matches class io.lettuce.core.RedisPublisher$SubscriptionCommand
2023-11-27 16:41:15,282 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation OnCancel: hasSuperType(erasure(name(equals(io.lettuce.core.protocol.RedisCommand)))) matches class io.lettuce.core.RedisPublisher$SubscriptionCommand
2023-11-27 16:41:15,293 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation OnComplete: (name(equals(complete)) and hasParameter(ofSize(0)) and not(isAbstract())) matches public void io.lettuce.core.RedisPublisher$SubscriptionCommand.complete()
2023-11-27 16:41:15,296 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation OnCancel: (name(equals(cancel)) and hasParameter(ofSize(0)) and not(isAbstract())) matches public void io.lettuce.core.RedisPublisher$SubscriptionCommand.cancel()
2023-11-27 16:41:15,298 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation OnCompleteExceptionally: (name(equals(completeExceptionally)) and hasParameter(hasTypes(erasures(containing(is(class java.lang.Throwable))))) and not(isAbstract())) matches public boolean io.lettuce.core.RedisPublisher$SubscriptionCommand.completeExceptionally(java.lang.Throwable)
2023-11-27 16:41:15,303 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation OnComplete: hasSuperType(erasure(name(equals(io.lettuce.core.protocol.RedisCommand)))) matches class io.lettuce.core.protocol.CommandWrapper
2023-11-27 16:41:15,303 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation OnCompleteExceptionally: hasSuperType(erasure(name(equals(io.lettuce.core.protocol.RedisCommand)))) matches class io.lettuce.core.protocol.CommandWrapper
2023-11-27 16:41:15,303 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation OnCancel: hasSuperType(erasure(name(equals(io.lettuce.core.protocol.RedisCommand)))) matches class io.lettuce.core.protocol.CommandWrapper
2023-11-27 16:41:15,387 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation OnComplete: (name(equals(complete)) and hasParameter(ofSize(0)) and not(isAbstract())) matches public void io.lettuce.core.protocol.CommandWrapper.complete()
2023-11-27 16:41:15,388 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation OnCancel: (name(equals(cancel)) and hasParameter(ofSize(0)) and not(isAbstract())) matches public void io.lettuce.core.protocol.CommandWrapper.cancel()
2023-11-27 16:41:15,390 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation OnCompleteExceptionally: (name(equals(completeExceptionally)) and hasParameter(hasTypes(erasures(containing(is(class java.lang.Throwable))))) and not(isAbstract())) matches public boolean io.lettuce.core.protocol.CommandWrapper.completeExceptionally(java.lang.Throwable)
2023-11-27 16:41:15,486 [http-nio-8080-exec-2] DEBUG co.elastic.apm.agent.bci.IndyPluginClassLoaderFactory - Creating plugin class loader for org.springframework.boot.loader.LaunchedURLClassLoader@579bb367 containing [co.elastic.apm.agent.lettuce.Lettuce34Instrumentation, co.elastic.apm.agent.lettuce.Lettuce5StopSpanInstrumentation$OnCompleteExceptionally$AdviceClass, co.elastic.apm.agent.lettuce.Lettuce34StopSpanInstrumentation$OnComplete, co.elastic.apm.agent.lettuce.Lettuce34StartSpanInstrumentation$AdviceClass, co.elastic.apm.agent.lettuce.Lettuce34StopSpanInstrumentation$OnCancel$AdviceClass, co.elastic.apm.agent.lettuce.Lettuce34StopSpanInstrumentation$OnCancel, co.elastic.apm.agent.lettuce.Lettuce5StopSpanInstrumentation$OnCompleteExceptionally, co.elastic.apm.agent.lettuce.Lettuce5StartSpanInstrumentation$AdviceClass, co.elastic.apm.agent.lettuce.Lettuce5StopSpanInstrumentation$OnComplete, co.elastic.apm.agent.lettuce.Lettuce34StopSpanInstrumentation$OnCompleteExceptionally, co.elastic.apm.agent.lettuce.Lettuce34StartSpanInstrumentation, co.elastic.apm.agent.lettuce.Lettuce34StopSpanInstrumentation, co.elastic.apm.agent.lettuce.Lettuce5StopSpanInstrumentation$OnCancel$AdviceClass, co.elastic.apm.agent.lettuce.Lettuce34StopSpanInstrumentation$OnCompleteExceptionally$AdviceClass, co.elastic.apm.agent.lettuce.Lettuce5StopSpanInstrumentation$OnComplete$AdviceClass, co.elastic.apm.agent.lettuce.Lettuce34StopSpanInstrumentation$OnComplete$AdviceClass, co.elastic.apm.agent.lettuce.Lettuce5StopSpanInstrumentation, co.elastic.apm.agent.lettuce.package-info, co.elastic.apm.agent.lettuce.Lettuce5StartSpanInstrumentation, co.elastic.apm.agent.lettuce.Lettuce5StopSpanInstrumentation$OnCancel, co.elastic.apm.agent.bci.classloading.LookupExposer]
2023-11-27 16:41:15,833 [lettuce-nioEventLoop-4-1] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation OnComplete: hasSuperType(erasure(name(equals(io.lettuce.core.protocol.RedisCommand)))) matches class io.lettuce.core.protocol.LatencyMeteredCommand
2023-11-27 16:41:15,833 [lettuce-nioEventLoop-4-1] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation OnCompleteExceptionally: hasSuperType(erasure(name(equals(io.lettuce.core.protocol.RedisCommand)))) matches class io.lettuce.core.protocol.LatencyMeteredCommand
2023-11-27 16:41:15,833 [lettuce-nioEventLoop-4-1] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation OnCancel: hasSuperType(erasure(name(equals(io.lettuce.core.protocol.RedisCommand)))) matches class io.lettuce.core.protocol.LatencyMeteredCommand

We have a lot of spring boot application, but this error only happens with a small part of all applications.

I tested with another agent versions, but I get the same error.

Can you help me find how this error happens and how we can fix it?

I can make the full log available.

Thanks.

There seems to be a problem with the spring reactor-core version this application is using in combination with our agent. Could you tell us the version of this library you are using?

Hello Jonas.

Sorry for late.

We're using spring boot two versions:

  • Spring boot 2.0.0-RELEASE with reactor-core-3.1.5-RELEASE.
  • Spring boot 2.0.9-RELEASE with reactor-core-3.1.16-RELEASE.

This issue happens in both.

When I exclude this class with argument -Delastic.apm.classes_excluded_from_instrumentation=reactor.core.* I see no error. Can I use this arg without problems?

Unfortunately those two versions are not supported:
Our supported technologies page states that spring-webflux is supported for spring version 5.2.3 or newer. Spring 5.2.3 ships reactor-core in version 3.3.2.RELEASE, which is the earliest supported version.

When I exclude this class with argument -Delastic.apm.classes_excluded_from_instrumentation=reactor.core.* I see no error. Can I use this arg without problems?

I would recommend you to rather disable the problematic instrumentation than relying on the package names. You can do this using the following option:

-Delastic.apm.disable_instrumentations=reactor,spring-webflux,springwebclient

Thanks for your answer, Jonas!

I'll apply this argument by default.

Have a good day.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.