I have the following test configuration. Basically I want to write the messages in either the fail or the notfail files depending on whether the grok filter would succeed. I've created the files proactively under a folder and all the folder and the files are owned by logstash user.
My problem is that only the _grokparsefailure entries gets written to the fail file and at the same time both successes and fails are output on my console.
What Am I missing. Why the successfully parsed evetns are only output on my console but not in the notfail file?
input
{
generator
{
message => "10.248.139.165 - - [30/Jun/2021:06:43:05 +0200] \"GET /favicon.ico HTTP/1.1\" 200 17542 \"http://10.21.47.143:8080/login?from=%2F\" \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36 Edg/91.0.864.59\""
count => 3
}
generator
{
message => "@s0.248864.59"
count => 3
}
}
filter
{
grok
{
keep_empty_captures => true
match =>
{ "message" => '(?<SourceIP>\d+\.\d+\.\d+\.\d+)[\s-]*\[(?<Date>\d{2}\/\w{3}\/\d{4}):(?<Time>\d{2}:\d{2}:\d{2})\s*\+\d{4}\]\s*\\"(?<Method>\w+)\s*(?<Resource>[-\w\s\/\.%\$]+)\\"\s*(?<ResponseCode>\d{3})\s*(?<Bytes>\d+)\s*\\"(?<MainURL>[\w\s\/\.:?=%]+)\\"\s*\\"(?<Agent>[\w\s\/\.();,]+)\\"'
}
add_field => { "LogCollector" => "daesoc01" }
}
}
output
{
if "_grokparsefailure" in [tags]
{
stdout { codec => rubydebug}
file
{
path => "/home/dimma/testfolder/lt/fail"
file_mode => 0600
}
}
else if "_grokparsefailure" not in [tags]
{
stdout { codec => rubydebug}
file
{
path => "home/dimma/testfolder/lt/notfail"
file_mode => 0600
}
}
}
