_Grokparsefailure even though pattern works in Grok Debugger

Elastic Stack Logstash
1

I am trying to parse IIS logs and I did test my pattern in the Grok Dubugger (inside Kibana) and it is working perfectly fine but logstash is giving me _grokparsefailure.

Here is my config file

input {
	file {
    type => "w3svc"
    start_position => "beginning"
    sincedb_path => "/dev/null"
    path => "/Users/samvidkulkarni/Desktop/Input/joust2-20180913_145201-w3svc2.txt"
  }
}

filter {
    if [type] == "w3svc"
    {
    grok {

    match => [ "message", "^%{TIMESTAMP_ISO8601:timestamp} %{NOTSPACE:s-ip} %{WORD:cs-method} %{NOTSPACE:cs-uri-stem} %{NOTSPACE:cs-uri-query} %{NUMBER:s-port} %{NOTSPACE:cs-username} %{NOTSPACE:c-ip} %{NOTSPACE:csUser-Agent} %{NOTSPACE:csReferer} %{NUMBER:sc-status} %{NUMBER:sc-substatus} %{NUMBER:sc-win32-status} %{NUMBER:time-taken}" ]  
   
   }


   geoip {
      add_tag => [ "ClientGeoIP" ]
      tag_on_failure => [ ]
      source => "c-ip"
      target => "c-ip"
      add_field => [ "[c-ip][coordinates]", "%{[c-ip][longitude]}" ]
      add_field => [ "[c-ip][coordinates]", "%{[c-ip][latitude]}" ]
    }

    geoip {
      add_tag => [ "ServerGeoIP" ]
      tag_on_failure => [ ]
      source => "s-ip"
      target => "s-ip"
      add_field => [ "[s-ip][coordinates]", "%{[s-ip][longitude]}" ]
      add_field => [ "[s-ip][coordinates]", "%{[s-ip][latitude]}" ]
    } 
   
 }
 }
	

output
{
	  if [type] == "w3svc" {
    elasticsearch {
      manage_template => false
      hosts => ["localhost:9200"]
      index => "w3svc-%{+YYYY.MM.dd}"
    }
    stdout { codec => rubydebug }

  }
}

Here is my output from IIS log file

2018-09-13 21:45:05 10.3.47.10 POST /EWS/Exchange.asmx &request_id=88de1f28-f7c8-4349-a85d-f169f12a9683 444 swfs\\tmsxe 10.4.47.11 TMSXE+5.5.0+(ExchangeServicesClient/15.00.0913.015) - 200 0 0 66

here is the output of logstash in console

{
          "path" => "/Users/samvidkulkarni/Desktop/Input/joust2-20180913_145201-w3svc2.txt",
          "tags" => [
        [0] "_grokparsefailure"
    ],
       "message" => "2018-09-13 21:45:05 10.3.47.10 POST /EWS/Exchange.asmx &request_id=88de1f28-f7c8-4349-a85d-f169f12a9683 444 swfs\\tmsxe 10.4.47.11 TMSXE+5.5.0+(ExchangeServicesClient/15.00.0913.015) - 200 0 0 66\r",
          "host" => "sam-MacBook-Air.local",
          "type" => "w3svc",
      "@version" => "1",
    "@timestamp" => 2018-09-19T18:37:13.798Z
}
2

In the documentation a "=>" is used after "message".
Also they use a "{" bracket instead of "[",

filter {
  grok {
    match => { "message" => "%{SYSLOGBASE} %{POSTFIX_QUEUEID:queue_id}: %{GREEDYDATA:syslog_message}" }
  }
}
3

thanks you sir. It works now.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.