Gathering RabbitMQ payload when it has a filed message

Elastic Stack Logstash
1

Hi All,

I'm trying to store the original RabbitMQ payload to orig_msg field with Logstash, but the payload itself contains a filed called message. Logstash add_field mutate { add_field => { "orig_msg" => "%{message}" } } crates the orig_msg filed with the value in the message field within the payload. How can I store the original payload in the orig_msg filed?

My Logstash pipeline:

input {
    rabbitmq {
        host => ["rabbitmq.xyz.local"]
        queue => "queue_1"
        heartbeat => 30
        durable => true
        password => "password"
        user => "user"
    }
}
filter {
    mutate {
        add_field => { "orig_msg" => "%{message}" }
    }

    if [@metadata][rabbitmq_properties][timestamp] {
        date {
            match => ["[@metadata][rabbitmq_properties][timestamp]", "UNIX"]
        }
    }

}
output {
    elasticsearch {
        hosts => "localhost:9200"
        index => "index_1"
        user => "elastic_user"
        password => "password"
    }

    stdout {
        codec => rubydebug { metadata => true }
    }
}

The test payload:
{"time":"2019-07-09T11:04:04.944+0200", "severity": "INFO", "service": "agent-test", "trace": "", "span": "", "parent": "", "exportable": "", "pid": "5760", "thread": "main", "class": "", "method": "", "evaluationTime": "", "message": "The following profiles are active: test", "params": "", "headers": ""}

Thanks.

2

The default codec in a rabbitmq input is json, so the json is parsed by the time the event is flushed to the pipeline. You could try changing the input codec to line or plain to prevent it parsing the data and then add a json filter.

3

And how can I do that? - Done (with Codec)
Thanks!

4

Add

codec => plain

to the options on the rabbitmq input.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.