Scripted fields not working on multiline Events

I am using multiline codec in my logstash config file in input section to merge few lines of log file. Lines 5,6,7,8,9 are getting merged into a single line and the same is visible in Kibana screen for message and sub_class fields(refer config file).
My requirement is to read the week consumption and month consumption and show it in new scripted field Consumption Data. I am not able to get any value for the merged line using scripted field.
I used below script and received ‘Value not present, for the merged line and ‘Value Present’ for all other lines.
Script -
if(doc['sub_class.keyword'].size()==0) return 'Value not present'; else return 'Value Present';

Log File -

  1. 2020/05/12_09:25:04,Attr 0x0000
  2. 2020/05/12_09:25:04, Attr 0x0000
  3. 2020/05/12_09:25:04, Attr 0x0000
  4. 2020/05/12_09:25:04, Attr 0x0000
  5. 2020/05/12_09:25:04, Daily Consumption report received
  6. 2020/05/12_09:25:04,Current week consumption was 100 (avail=1)
  7. 2020/05/12_09:25:04,Day is not Monday, so add current day into current week
  8. 2020/05/12_09:25:04,Current month consumption was 5000 (avail=1)
  9. 2020/05/12_09:25:04,Day is not Monday, so add current day into current week
  10. 2020/05/12_09:25:04, Attr 0x0000
  11. 2020/05/12_09:25:04, Attr 0x0000
  12. 2020/05/12_09:25:04, Attr 0x0000

Logstash Config file -
input {
file {
path => ["---------------------------------/WHOHealth.txt"]
start_position => "beginning"
sincedb_path => "NUL"
codec => multiline {
pattern => "(^.+Current week .+)|(^.+Day is .+)|(^.+Current month.+)”
what => "previous"
negate => "false"
}
}
}
filter {
if ([message] =~ "{"index") {
drop {}
} else if ([message] =~ "month") {
grok {
match => {"message" => "%{GREEDYDATA:log_line}"}
}
mutate {
add_field => { "sub_class" => "%{[log_line]}" }
}
}

}
output {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "TestIndex"
}
}

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.