Separated conf file duplicates search

[sud0 (Luke)]

Hi.

I've created two conf files under /etc/logstash/conf.d. Because I have two businesses in two different countries, I want them to be displayed in two separately indexes.

• Index name nz-*
• Index name au-*

However, in Kibana when discovering the data, even if I select only index nz-*, I can still see data from the other one.

What am I doing wrong?

business1.conf:

output {
    elasticsearch {
        hosts => ["localhost:9200"]
        # Weekly index (for pruning)
        index => "au-log-index-%{+YYYY.'w'ww}"
    }
    stdout { codec => rubydebug }
}

business2.conf:

output {
    elasticsearch {
        hosts => ["localhost:9200"]
        # Weekly index (for pruning)
        index => "nz-log-index-%{+YYYY.'w'ww}"
    }
    stdout { codec => rubydebug }
}

[Badger]

If both configurations are running in the same pipeline then every event will get written to both elasticsearch outputs. You need to either run each business in its own pipeline or use conditionals in your processing.

[sud0 (Luke)]

That's interesting. Thanks for the help! I'll have a read about pipelines for sure! Will mark this question as solved.