How to queue ECS formatted logs through RabbitMQ

[bvoros (Bertalan Voros)]

Hello all,

Our logging infrastructure is the following:

log shippers -> logstash -> rabbitmq -> logstash -> elasticsearch

I am trying to start using ECS, have the template set up. However, when the first logstash places the log document in the RabbitMQ queue, the original ecs log message gets embedded inside another json document.

Example: (incomplete)
{"@timestamp":"2023-06-02T14:56:14.094Z","message":"{"@timestamp":"2023-06-02T17:56:12.8391096+03:00","log.level":"Information","message":"Simple test log","ecs.version":"8.4.0"

Is it possible to configure logstash to simply pass through the document as received?

Thanks in advance,
BV

[leandrojmp (Leandro Pereira)]

@bvoros

Is it possible to configure logstash to simply pass through the document as received?

You may try to change the codec in your output.

I do not use rabbitmq, but I have the following codec configuration on some Kafka outputs to send the original message that Logstash received.

codec => plain { format => "%{message}" }

[bvoros (Bertalan Voros)]

Thank you, this is a good tip, will try and report back here.

[bvoros (Bertalan Voros)]

Thanks again for the tip, the following appears to be working when added to the rabbitmq output plugin.

codec => plain {
        format => "%{message}"
        ecs_compatibility => v8
      }