grokParseFailure but works on grokdebug tester

Elastic Stack Logstash
1

Hi, I'm on Logstash 5.4.3 (the latest version to support the community NewRelic plugin), shipping IIS logs from filebeat.

I've got log entries which pass the grok filter at grokdebug.herokuapp.com, but fail to parse on the box.

Maybe somebody could point out my schoolboy error to me :wink:

Here's my logstash conf:

input {
  beats { port => 5044   }
}

filter {
  if [message] =~ "^#" { drop {} }

  if [type] == "iis" {
    grok {
      match => { 
	   "message" => 
			"%{TIMESTAMP_ISO8601:log_timestamp} %{WORD:S_sitename} %{WORD:S_computername} %{IP:S_ip} %{WORD:CS_method} %{URIPATH:CS_uri_stem} %{NUMBER:S_port} %{NOTSPACE:CS_username} %{IPORHOST:C_ip} %{NOTSPACE:CS_version} %{NOTSPACE:CS_useragent} %{NOTSPACE:SC_host} %{NUMBER:SC_status} %{NUMBER:SC_substatus} %{NUMBER:SC_win32_status} %{NUMBER:SC_bytes} %{NUMBER:CS_bytes} %{NUMBER:time_taken}"
			
	   "source" => "%{GREEDYDATA:log_source}"
	  }
    }
  }

  if [SC_status] == "200" { drop { } }
}

filter {
  grok {
    remove_field => [ "message", "offset", "input_type", "type", "tags", "@timestamp", "@version", "beat", "host", "CS_username", "CS_version", "CS_useragent", "CS_cookie", "cs_referer", "S_port", "SC_substatus", "SC_win32_status", "SC_bytes", "CS_bytes" ]
  }
}

output {
  if "_grokparsefailure" in [tags] {
    file { "path" => "C:\logstash\logs\grok_failures.txt" }
  } else {
    file { "path" => "C:\logstash\logs\log.txt" }
    newrelic {
      account_id => 123456
      insert_key => mykey
    }
  }
}

and here's an example of a record that passes in the tester but fails in logstash:

in the grok_failure file

{"log_timestamp":"2017-11-16 17:46:03","S_sitename":"W3SVC1","SC_bytes":"41679","source":"C:\\inetpub\\logs\\LogFiles\\W3SVC1\\u_ex171116.log","type":"iis","SC_substatus":"0","SC_host":"stage-webcd02.mysite.co.uk","CS_bytes":"812","@version":"1","beat":{"hostname":"UK02SWEBCD02","name":"UK02SWEBCD02","version":"5.6.2"},"host":"UK02SWEBCD02","SC_win32_status":"0","SC_status":"404","offset":276092,"CS_version":"HTTP/1.1","input_type":"log","message":"2017-11-16 17:46:03 W3SVC1 UK02SWEBCD02 10.248.11.5 GET /amble8 443 - 192.168.50.180 HTTP/1.1 Mozilla/5.0+(Windows+NT+6.1;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/62.0.3202.94+Safari/537.36 stage-webcd02.mysite.co.uk 404 0 0 41679 812 781","time_taken":"781","tags":["beats_input_codec_plain_applied","_grokparsefailure"],"CS_method":"GET","@timestamp":"2017-11-16T17:46:54.029Z","CS_useragent":"Mozilla/5.0+(Windows+NT+6.1;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/62.0.3202.94+Safari/537.36","S_ip":"10.248.11.5","CS_uri_stem":"/amble8","S_computername":"UK02SWEBCD02","S_port":"443","CS_username":"-","C_ip":"192.168.50.180"}

and the IIS log as shipped by filebeat:

2017-11-16 17:46:03 W3SVC1 UK02SWEBCD02 10.248.11.5 GET /amble8 443 - 192.168.50.180 HTTP/1.1 Mozilla/5.0+(Windows+NT+6.1;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/62.0.3202.94+Safari/537.36 stage-webcd02.mysite.co.uk 404 0 0 41679 812 781
2

You're getting a _grokparsefailure tag but the grok filter appears to be successful since the fields have been extracted. I suspect it's the second grok filter that's causing this. Use a mutate filter for unconditionally removing fields.

3

Thanks, that works.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.