Java Agent not showing RMQ while using SimpleRabbitListenerContainerFactory

Kibana version: 7.9.3

Elasticsearch version: 7.9.3

APM Server version: 7.9.3

APM Agent language and version: Java 1.22

Browser version: -

Original install method: Cloud ELK

Description of the problem including expected versus actual behavior:

Hi!
We try to get insights into producing and consuming messages via RMQ and therefore updated to Agent version 1.22 but are missing any output related to that. REST requests that result in producing a message are monitored and shown but we are not able to see the call to the producer nor the consumer altough messages are consumed and we see the result (updates in DB).
We use a customized SimpleRabbitListenerContainerFactory maybe it is related to that and someone could give us a hint how to get it running.

Thanks in advance
Rainer

Rabbit MQ configuration:

@EnableRabbit
@Configuration
public class RabbitMqConfig {

    ...

    @Bean
    public RetryOperationsInterceptor listenerRetryInterceptor() {
        return RetryInterceptorBuilder.stateless()
            .backOffOptions(1000, 3.0, 10000)
            .maxAttempts(3)
            .recoverer(new ImmediateRequeueMessageRecoverer())
            .build();
    }

    @Bean
    public SimpleRabbitListenerContainerFactory listenerRetryContainerFactory(
        ConnectionFactory connectionFactory, RetryOperationsInterceptor listenerRetryInterceptor) {
        SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
        factory.setConnectionFactory(connectionFactory);

        Advice[] adviceChain = { listenerRetryInterceptor };
        factory.setAdviceChain(adviceChain);

        return factory;
    }
}

Listener:

@Component
public class SubOrderStatusUpdateEventListener {

      ....

    @RabbitListener(queues = "${mq.listener.suborder-status-update.queue}", containerFactory = "listenerRetryContainerFactory")
    public void consumeBlocking(Message incomingMessage) throws Exception {
        Stopwatch sw = Stopwatch.createStarted();

        EventListenerResult serviceResult = this.eventListenerService.updateSubOrderStatus(incomingMessage);

        if (serviceResult.isTemporaryError()) {
            throw new Exception("temporary error, retry later");
        } else if (serviceResult.isPermanentError()) {
            putIntoDeadLetterQueue(incomingMessage);
        }

        sw.stop();

        StructuredLogger.logInfoMessage(LogMessage.PROCESSING_TIME_FOR_MESSAGE, Map.of("TIME_MILLIS", Long.toString(sw.elapsed(TimeUnit.MILLISECONDS)), "MESSAGE_ID", retrieveMessageId(incomingMessage).orElse("noMessageId")));
    }
}

Hi @rainer-funk ,

From what I understand, you only have transactions created from HTTP requests, but the following are missing:

  • spans when sending outgoing messages to RabbitMQ from those HTTP requests
  • transactions that should be created when consuming those RabbitMQ messages

Few questions here:

  • What are your RabbitMQ / Spring dependencies versions here ?
  • Do you see RabbitMQ classes instrumented in the agent log when log_level=debug ?
  • could you set a breakpoint in the consumeBlocking method and provide us the stack trace ?

Hi Sylvain,

yes you understand it right.

We are using AMQP client in version 5.9.0 and Springt Boot Starter for AMQP in version 2.4.3.

Will come back later with a stacktrace

Hi @rainer-funk did you get a chance to capture

  • agent logs with log_level=debug
  • call trace when setting a breakpoint in consumeBlocking method ?

Getting those would definitely help to better understand the source of your issue here.
Thanks!

hi @rainer-funk , @Sylvain_Juge
I will deal with this issue later today

I was unable to reproduce the problem, I created similar beans and used spring-boot-amqp version 2.4.3.
Are you sure the given consumer is reading the message? Can you add log info to the top of your consumeBLocking method and send the full application log with the agent logs with log_level = debug setting enabled.
p.s. earlier I wrote post about spring amqp monitoring. It may be useful.

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.