Filtering using grok

Elastic Stack Logstash
1

Hi,

I want to perform different operation based on the entry in my log file.

2016-05-12 18:32:11,452 [http-8080-2] INFO AwsLifeCycleManager Aws VMProvisioning SUCCESS AssetID IP AssetName

If the operation is FAILED instead of SUCCESS then the log entry will be

2016-05-12 18:32:11,452 [http-8080-2] INFO AwsLifeCycleManager Aws VMProvisioning FAILED ERROR MESSAGE

Please help me to filter this logfile based on SUCCESS/FAILED using grok, so that I can list the successful operations and failed operations separately.

Thanks in advance .

2

First up you need a grok pattern, try http://grokdebug.herokuapp.com/ for assistance on that.
Once you have the status field defined you can do the next step you're after.

1 Like
3

Thanks Mark ,

I have already created a grok pattern for matching the log statements,
grok {
match => {"message" => "(?%{YEAR}[/-]%{MONTHNUM}[/-]%{MONTHDAY}) %{TIME:time} (?[\w{4}-\d{1,}-\d{1,}]) %{WORD:info} %{WORD:cls} %{WORD:provider} %{WORD:method} %{WORD:result} "}
if [result] == "FAILED" {
match => {"message" => " %{WORD:ERROR} %{GREEDYDATA:message}" }
}
else {
match => {"message" => " %{WORD:asset} %{IP:hostip} %{GREEDYDATA:privatename}" }
}
}

Could you Please tell me what is wrong in this condition statement

Thanks.

4

You can't have a conditional inside a grok filter. I suggest you use two grok expressions that Logstash will try in order:

grok {
  match => {
    "message" => [
      "(?%{YEAR}[/-]%{MONTHNUM}[/-]%{MONTHDAY}) %{TIME:time} (?[\w{4}-\d{1,}-\d{1,}]) %{WORD:info} %{WORD:cls} %{WORD:provider} %{WORD:method} (?<result>SUCCESS) %{WORD:asset} %{IP:hostip} %{GREEDYDATA:privatename}",
      "(?%{YEAR}[/-]%{MONTHNUM}[/-]%{MONTHDAY}) %{TIME:time} (?[\w{4}-\d{1,}-\d{1,}]) %{WORD:info} %{WORD:cls} %{WORD:provider} %{WORD:method} (?<result>FAILED) %{WORD:ERROR} %{GREEDYDATA:message}"
    ]
  }
}
1 Like
5

Thanks Magnus,
It worked :slight_smile: