HI All,
I have a log file like the following which I use logstash to process.
FilterPerfLogging input_count:2 ,TimeTaken:0.272 ,ConfigRate:7.35294 ,output_count:2
SubFilterLogging ContraintSize:5, ,TimeTaken:0.501
Now these two keywords FilterPerfLogging , SubFilterLogging are two different events with different parameters which I use KV filter to extract and store in elasticsearch.
My problem is that I am not able to store them separately in different Indexes,
This is my conf file
input {
file {
path => "C:/Temp1/*.syslog"
start_position => beginning
ignore_older => 0
}
}
filter {
if [message] !~ /FilterPerfLogging / and [message] !~ /SubFilterLogging / {
drop { }
}
if [message] == /FilterPerfLogging / {
kv {
value_split => ":"
field_split => " ,"
}
mutate {
convert => {
"input_count" => "integer"
"output_count" => "integer"
"TimeTaken" => "float"
"ConfigRate" => "float"
}
}
}
if [message] == /SubFilterLogging / {
kv {
value_split => ":"
field_split => " ,"
}
mutate {
convert => {
"ContraintSize" => "integer"
}
add_tag => ["subfilter"]
}
}
}
output {
if "subfilter" in [tags] {
elasticsearch {
index => "subFilterIndex"
}
}
else{
elasticsearch {
}
}
}
My problem is the index subFilterIndex is never created and every entry goes to the default index.
And the fields which were earlier created by the KV filter is also gone and all I see is the message string in kibana.
Can somebody suggest some way or figure out what I have been missing.
Thanks,
Ayush