_grokparsefailure tag errors parsing IIS logs using FileBeat

[hp-adl (Hoc Phung)]

Hello,

I've been struggling for 2 solid days on this and I just can't work out what keeps causing these _grokparsefailure errors. I'm using the following services and versions for my ELK stack.

FileBeat 5.1.1 (Windows)
Logstash 5.1.1 (Ubuntu)
AWS Elasticsearch Service 2.3 + Kibana

Here's the output in Kibana

I have configured IIS to log one file per server and logging everything.

Here's is my logstash conf file

input {
  beats {
    port => 5044
    type => iis
  }
}

filter {
  if [message] =~ "^#" {
    drop {}
  }

  if [type] == "iis" {
    grok {
      match => ["message","%{TIMESTAMP_ISO8601:log_timestamp} %{WORD:serviceName} %{WORD:serverName} %{IP:serverIP} %{WORD:method} %{NOTSPACE:uriStem} %{NOTSPACE:uriQuery} %{NUMBER:port} %{WORD:username} %{IP:clientIP} %{WORD:protocolVersion} %{WORD:userAgent} %{WORD:cookie} %{WORD:referer} %{WORD:requestHost} %{NUMBER:response} %{NUMBER:subresponse} %{NUMBER:win32response} %{NUMBER:bytesSent} %{NUMBER:bytesReceived} %{NUMBER:timetaken}"]
  }

  date {
    match => [ "log_timestamp", "YYYY-MM-dd HH:mm:ss" ]
      timezone => "Etc/UTC"
  }

  if [bytesSent] {
    ruby {
      code => "event['kilobytesSent'] = event['bytesSent'].to_i / 1024.0"
    }
  }

  if [bytesReceived] {
    ruby {
      code => "event['kilobytesReceived'] = event['bytesReceived'].to_i / 1024.0"
    }
  }

  mutate {
    convert => { "bytesSent" => "integer" }
    convert => { "bytesReceived" => "integer" }
    convert => { "timetaken" => "integer" }
    remove_field => [ "log_timestamp" ]
    replace => { "log_timestamp" => "%{log_timestamp}.000" }
  }

    useragent {
        source => "useragent"
        prefix => "browser"
    }
  }
}

output {
  if [type] == "iis" and "_grokparsefailure" in [tags] {
    file { path => "/var/log/logstash/failed_iis_events-%{+YYYY-MM-dd}.log" }
  }
}

    output {
      elasticsearch {
        hosts => ["search-grp-es-demo-hpsswdyis6j23f7dg2f7wwpq3i.ap-southeast-2.es.amazonaws.com:80"]
        index => "%{type}-%{+YYYY.MM.dd}"
        template => "/etc/logstash/conf.d/templates/iis-template.json"
        template_name => "iis"
        document_type => "iis"
        template_overwrite => true
        manage_template => true
      }
    }

Any help would be much appreciated!

[magnusbaeck (Magnus Bäck)]

You'll want to change %{WORD:serverName} to e.g. %{NOTSPACE:serverName} because WORD doesn't include hyphens (IIRC).

But in general, never try the full expression at once. Start with the smallest possible expression, in this case %{TIMESTAMP_ISO8601:log_timestamp}, and gradually add more and more until things stop working. Then you know that the most recent addition was wrong.

[hp-adl (Hoc Phung)]

Changing %{WORD:serverName} to %{NOTSPACE:serverName} did the trick! If I was better with my RegEx I would have picked that up quicker. I also fixed the other fields, replace WORD with NOTSPACE. Shame the _grokparsefailure doesn't tell you much.

Thanks for your prompt assistance Magnus. Elastic make great software!