Two outputs with different data

Elastic Stack Logstash
1

I want to take one input, do some filters, send output to one ES index, do more filters, then send to a second ES index.

The result should be one index with a set of fields, and the second index has a mostly different set of fields but based on the same original input.

In my logstash config, I tried input -> filter -> output -> filter -> output, where the second filter removes most fields and adds a few new ones before sending it to the second index, but the result is both indexes end up getting the combined filter result.

Any ideas?

2

Events are read from inputs, sent through all the filters, then written to the outputs. So what you are seeing is expected.

You could write the events to a second logstash instance, or a different pipeline if you are using a recent version.

3

You can use the clone filter to create a second event. Add the additional fields only to one of these events and then send them to different outputs.

4

That's a great idea. I'll try that and see if I can get it working. Thanks!

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.