How to queue ECS formatted logs through RabbitMQ

Elastic Stack Logstash
1

Hello all,

Our logging infrastructure is the following:

log shippers -> logstash -> rabbitmq -> logstash -> elasticsearch

I am trying to start using ECS, have the template set up. However, when the first logstash places the log document in the RabbitMQ queue, the original ecs log message gets embedded inside another json document.

Example: (incomplete)
{"@timestamp":"2023-06-02T14:56:14.094Z","message":"{"@timestamp":"2023-06-02T17:56:12.8391096+03:00","log.level":"Information","message":"Simple test log","ecs.version":"8.4.0"

Is it possible to configure logstash to simply pass through the document as received?

Thanks in advance,
BV

2

You may try to change the codec in your output.

I do not use rabbitmq, but I have the following codec configuration on some Kafka outputs to send the original message that Logstash received.

codec => plain { format => "%{message}" }
1 Like
3

Thank you, this is a good tip, will try and report back here.

4

Thanks again for the tip, the following appears to be working when added to the rabbitmq output plugin.

codec => plain {
        format => "%{message}"
        ecs_compatibility => v8
      }
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.