Logs going to the wrong index

Elastic Stack Logstash
1

Hi ,

I have three logstash config files 00_inputs.conf, 01_app1.conf, 02_app2.conf.
01_inputs.conf just defines the beats input and thats it, nothing more.

02_app1.conf

filter {
if [fields][index] == "app_fischer" {
mutate {
replace => {
"[@metadata][index]" => "%{[fields][index]}"
}
}
}
}
output {
elasticsearch {
hosts => "http://elasticco-qa-e.domain.com:80"
#manage_template => false
index => app_fischer
user => elastic
password => changeme
}
stdout { codec => rubydebug}

01_app.conf

filter {
if [fields][source] == "app_tomcat_perf" {

                  grok {
                       match => {"message" => "%{MONTHDAY} %{MONTH} %{YEAR} %{TIME},%{NUMBER:duration} %{WORD:loglevel}  %{WORD:Activity}   \[\{%{DATA:foo1}\}\]:(.*) execution time: %{NUMBER:executionTime:float} ms"}
        }

}
output {
elasticsearch {
hosts => "http://elasticco-qa-e.domain.com:80"
#manage_template => false
index => harmonic
user => elastic
password => changeme
}
stdout { codec => rubydebug}
}

But i see a strange behaviour, the logs are going to the wrong index. I mean the logs which should be going to the index defined in 02_app1.conf are actually going to the index defined in 01_app.conf .

When i search the elastic search through GET , i can see that _index has the mismatch. Need to know what could be the reason. Is it that the ordering of the files is wrong and which ever file is first ?

2

With your configuration all events will go to both the harmonic and app_fischer indexes. You'll have to wrap your elasticsearch outputs in conditionals (similar to your filters) to get another behavior.

3

Thanks Magnus, it worked.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.