# grokParseFailure but works on grokdebug tester

**URL:** <https://discuss.elastic.co/t/grokparsefailure-but-works-on-grokdebug-tester/108101>\
**Category:** Logstash\
**Created:** [November 17, 2017, 9:59am UTC](https://discuss.elastic.co/t/grokparsefailure-but-works-on-grokdebug-tester/108101 "2017-11-17T09:59:55Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![richard-lee-bjss](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richard-lee-bjss/32/24377_2.png) [@richard-lee-bjss](https://discuss.elastic.co/u/richard-lee-bjss)\
**Post date:** [November 17, 2017, 9:59am UTC](https://discuss.elastic.co/t/grokparsefailure-but-works-on-grokdebug-tester/108101/1 "2017-11-17T09:59:55Z")

</div>

Hi, I'm on Logstash 5.4.3 (the latest version to support the community NewRelic plugin), shipping IIS logs from filebeat.

I've got log entries which pass the grok filter at [grokdebug.herokuapp.com](http://grokdebug.herokuapp.com), but fail to parse on the box.

Maybe somebody could point out my schoolboy error to me 😉

Here's my logstash conf:

```auto
input {
  beats { port => 5044 }
}

filter {
  if [message] =~ "^#" { drop {} }

  if [type] == "iis" {
    grok {
      match => { 
	   "message" => 
			"%{TIMESTAMP_ISO8601:log_timestamp} %{WORD:S_sitename} %{WORD:S_computername} %{IP:S_ip} %{WORD:CS_method} %{URIPATH:CS_uri_stem} %{NUMBER:S_port} %{NOTSPACE:CS_username} %{IPORHOST:C_ip} %{NOTSPACE:CS_version} %{NOTSPACE:CS_useragent} %{NOTSPACE:SC_host} %{NUMBER:SC_status} %{NUMBER:SC_substatus} %{NUMBER:SC_win32_status} %{NUMBER:SC_bytes} %{NUMBER:CS_bytes} %{NUMBER:time_taken}"
			
	   "source" => "%{GREEDYDATA:log_source}"
	  }
    }
  }

  if [SC_status] == "200" { drop { } }
}

filter {
  grok {
    remove_field => ["message", "offset", "input_type", "type", "tags", "@timestamp", "@version", "beat", "host", "CS_username", "CS_version", "CS_useragent", "CS_cookie", "cs_referer", "S_port", "SC_substatus", "SC_win32_status", "SC_bytes", "CS_bytes"]
  }
}

output {
  if "_grokparsefailure" in [tags] {
    file { "path" => "C:\logstash\logs\grok_failures.txt" }
  } else {
    file { "path" => "C:\logstash\logs\log.txt" }
    newrelic {
      account_id => 123456
      insert_key => mykey
    }
  }
}

```

and here's an example of a record that passes in the tester but fails in logstash:

in the grok\_failure file

```auto
{"log_timestamp":"2017-11-16 17:46:03","S_sitename":"W3SVC1","SC_bytes":"41679","source":"C:\\inetpub\\logs\\LogFiles\\W3SVC1\\u_ex171116.log","type":"iis","SC_substatus":"0","SC_host":"stage-webcd02.mysite.co.uk","CS_bytes":"812","@version":"1","beat":{"hostname":"UK02SWEBCD02","name":"UK02SWEBCD02","version":"5.6.2"},"host":"UK02SWEBCD02","SC_win32_status":"0","SC_status":"404","offset":276092,"CS_version":"HTTP/1.1","input_type":"log","message":"2017-11-16 17:46:03 W3SVC1 UK02SWEBCD02 10.248.11.5 GET /amble8 443 - 192.168.50.180 HTTP/1.1 Mozilla/5.0+(Windows+NT+6.1;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/62.0.3202.94+Safari/537.36 stage-webcd02.mysite.co.uk 404 0 0 41679 812 781","time_taken":"781","tags":["beats_input_codec_plain_applied","_grokparsefailure"],"CS_method":"GET","@timestamp":"2017-11-16T17:46:54.029Z","CS_useragent":"Mozilla/5.0+(Windows+NT+6.1;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/62.0.3202.94+Safari/537.36","S_ip":"10.248.11.5","CS_uri_stem":"/amble8","S_computername":"UK02SWEBCD02","S_port":"443","CS_username":"-","C_ip":"192.168.50.180"}

```

and the IIS log as shipped by filebeat:

```auto
2017-11-16 17:46:03 W3SVC1 UK02SWEBCD02 10.248.11.5 GET /amble8 443 - 192.168.50.180 HTTP/1.1 Mozilla/5.0+(Windows+NT+6.1;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/62.0.3202.94+Safari/537.36 stage-webcd02.mysite.co.uk 404 0 0 41679 812 781

```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 17, 2017, 12:10pm UTC](https://discuss.elastic.co/t/grokparsefailure-but-works-on-grokdebug-tester/108101/2 "2017-11-17T12:10:03Z")

</div>

You're getting a `_grokparsefailure` tag but the grok filter appears to be successful since the fields have been extracted. I suspect it's the second grok filter that's causing this. Use a mutate filter for unconditionally removing fields.

---

<div class="post-metadata">

**Author:** ![richard-lee-bjss](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richard-lee-bjss/32/24377_2.png) [@richard-lee-bjss](https://discuss.elastic.co/u/richard-lee-bjss)\
**Post date:** [November 17, 2017, 12:25pm UTC](https://discuss.elastic.co/t/grokparsefailure-but-works-on-grokdebug-tester/108101/3 "2017-11-17T12:25:12Z")

</div>

Thanks, that works.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 15, 2017, 12:25pm UTC](https://discuss.elastic.co/t/grokparsefailure-but-works-on-grokdebug-tester/108101/4 "2017-12-15T12:25:14Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.