I am trying to restrict content/data to go to elastic search if grok not able to parse data in input.log but seems if "_grokparsefailure" not in [tags] not working. I am parsing some input log and matching some pattern using grok. If grok does not find a match, this match is still going to output and I can see unmatched (_grokparsefailure) text in tags in elastic search indexed docs. So i don't want any data to be passed to elastic search if grok pattern fails. Hope my question is clear.
My logstash.conf file.
input {
file {
path => "/opt/elasticSearch/logstash-1.4.2/input.log"
codec => multiline {
pattern => "^["
negate => true
what => previous
}
start_position => "end"
}
}
filter {
grok {
match => [
"message", "^[%{GREEDYDATA}] %{GREEDYDATA} Searching hotels for country %{GREEDYDATA:country}, city %{GREEDYDATA:city}, checkin %{GREEDYDATA:checkin}, checkout %{GREEDYDATA:checkout}, roomstay %{GREEDYDATA:roomstay}, No. of hotels returned is %{NUMBER:hotelcount} ."
]
}
}
output {
if "_grokparsefailure" not in [tags]{
elasticsearch {
cluster => "elasticsearchdev"
}
}
}