We're having an issue with Logstash and/or Elasticsearch, and we're not really sure where the issue resides. We are running Logstash and Elasticsearch on a single physical server and when viewing the data in Kibana, two indices incorrectly contain the same data. Our intent is to separate the data into two separate Elasticsearch indices, but we have so far been unsuccessful.
The local Logstash instance obtains log data from two separate files on the filesystem as configured by two separate Logstash configuration files located in /etc/logstash/conf.d. Each of the config files calls the elasticsearch plug-in using index => "logstash-indexname1" (in the first config file) and index => "logstash-indexname2" in the second config file. Of course in the "file" section, the required path variable is set accordingly and verified correct. The data are input to Elasticsearch, but unfortunately they are incorrectly merged into both indices when they should be separate.
As an aside, Logstash is started with systemd using ExecStart=/usr/share/logstash/bin/logstash --quiet -f /etc/logstash/conf.d --path.settings /etc/logstash
We wanted to ask the community what might cause this behavior.
Thanks in advance.