Here's the back story, I'm setting up a logstash server that will take data from file beats and ship it to the elastic cloud. I originally created a configuration file that would use a beats input, no filters and then output to elastic search. To test the connectivity I added the http input, under the beats and fired it up. I was able to send data to elasticsearch (via REST) but I didn't like the index scheme it applied, as I wrote it to leverage the metadata with beats, so I created a separate configuration file that only had the http input and it wrote to an elasticsearch output with a different index and type pattern definition. Essentially, two configuration files very similar setups, different inputs.
Now I have two configuration files, one that defines a beats input, and one that defines the http input. However, any document I send to the http input will get indexed under the index patterns that were defined in both of the configuration files, so they are getting indexed twice.
Not sure if this is enough data to describe the problem, but the configurations are very simple.
Is this expected behavior? Even though they are defined in separate configuration files, all data that comes into logstash inputs will be written to all defined outputs, regardless of being in multiple files? That is what I'm experiencing. I thought I saw something about using conditionals to possibly solve this but couldn't find anything that elaborated in detail.