Configure Grok Filters

[shirm]

Hi All, I am just starting out with Logstash.
I am trying to parse a logfile with error contents as

2018-09-03 15:31:51.9682 Error ActiveDirectory Application MyApp is referenced by groups uADGROUP1 and uADGROUP2 - only permissions for uADGROUP1 will apply

I have written a logstash config as below

# contents of logstash\bin\logstash.config

# input
input {
file {
    path => "C:\DATA\packagepromoterdata\diagnostics\ActiveDirectoryTest.log"
    start_position => beginning        
    }
}

filter {
grok {
    match => { "message" => "Error\sActiveDirectory\sApplication\s%{WORD:App_Name}\sis\sreferenced\sby\sgroups\s%{WORD:AD_Group1}\sand\s%{WORD:AD_Group2}\s-\sonly\spermissions\sfor\s%{WORD:ADGroup_Apply}\swill\sapply"}        
    }
}

output {
  stdout {  }
}

I have tried this filter in Grok Debugger: http://grokdebug.herokuapp.com/ and confirmed that this filter works, however while trying it in the config file, all the entries in the log entries are getting displayed

[AquaX (Andreas Helmer)]

If your grok pattern does not match what you want then you should drop the message.
Add this line after your grok filter inside the filter{} section

if "_grokparsefailure" in [tags] {
drop {}
}

Should look like this:

# contents of logstash\bin\logstash.config

# input
input {
file {
    path => "C:\DATA\packagepromoterdata\diagnostics\ActiveDirectoryTest.log"
    start_position => beginning        
    }
}

filter {
grok {
          match => { "message" => "Error\sActiveDirectory\sApplication\s%{WORD:App_Name}\sis\sreferenced\sby\sgroups\s%{WORD:AD_Group1}\sand\s%{WORD:AD_Group2}\s-\sonly\spermissions\sfor\s%{WORD:ADGroup_Apply}\swill\sapply"}        
    }

    if "_grokparsefailure" in [tags] {
          drop {}
    }
}

output {
  stdout {  }
}