Duplicate Events captured in Indexes

Elastic Stack Logstash
1

I have 2 indexes named MyIndex1 and MyIndex2.

I have 2 different conf files targeting multiple paths. Both conf has different paths completely. These conf at the end has individual index names. The only common thing is that both the conf files are present under a single folder named Configurations.
I have used logstash -f Configurations as the command line to use all the conf's present under the folder.
The problem which I am seeing here is that, events from these paths are captured successfully...but it is present in both the indexes.

Am I missing something or made a mistake ?

2

When you have several conf files, logstash marges them and consider it as a unique conf file. So if your output only changes according to the file (and not other filter), then it's normal.

What you can do is, if for example, input is file type, to add a different tag in the input and then filter the output index accordingly to the tag

3

I have now added the type field as a IF check, to capture things in appropriate indexes. Thanks !!!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.