Send only one field in pipeline to pipeline communication

Elastic Stack Logstash
#1

Hello,
I'd like my 1st pipeline to send the event to an Elasticsearch output and only a field's event to the input of a 2nd pipeline. Is it possible to send only one field in a pipeline and not a complete event?

#2

Remove those extra fields using mutate filter.

#3

Hello, thank you for your reply. I can't remove them because I need all field for my elasticsearch output.

#4

The pipeline to pipeline using the pipeline output does not support the codec option to change the format of the output message, which would allow you to send just one field to a pipeline and the entire message to the other pipeline.

To do this you will need to duplicate your event using the clone filter, the prune filter and some conditionals to change only the event that was cloned.

Something like this:

filter {
    clone {
        clones => ["cloned"]
    }
    if [type] == "cloned" {
        prune {
            whitelist_names => [ "field-you-want-to-send" ]
        }
    }
}
output {
    if [type] == "cloned" {
        pipeline { your pipeline-to-pipeline}
    }
    if [type] != "cloned" {
        your other output
    }
}
#5

If you have two pipelines then just remove the fields (using mutate or prune) in one of them. The other one will send the complete event to elasticsearch. This is the forked-path pattern for pipeline-to-pipeline communication.

#6

Thank you for your answers.