Add date field from IIS-log

Elastic Stack Logstash
1

Hi, i'm currently trying to add an additional date field for when the log was created called log_timestamp. But the Date filter does not seem to convert log_timestamp into a date type.
My config is below.

  filter {
      if [type] == 'iis_log' {
        if [message] =~ "^#" {
          drop {}
        }
        grok {
          match => { "message" => "%{TIMESTAMP_ISO8601:log_timestamp} %{IPORHOST:hostip} %{WORD:method} %{URIPATH:page} %{NOTSPACE:query} %{NUMBER:port:int} %{NOTSPACE:username} %{IPORHOST:clientip} %{NOTSPACE:useragent} %{NOTSPACE:cs-host} %{NUMBER:status:int} %{NUMBER:response:int} %{NUMBER:win32status:int}  %{NUMBER:timetaken:int}" }
        }
        geoip {
          source => "clientip"
          target => "geoip"
        }
        date {
          match => [ "log_timestamp", "YYYY-MM-dd HH:mm:ss" ]
          target => "log_timestamp"
        }
      }
    }
2

Please give an example of a log_timestamp value that isn't converted as you expect.

3

Thank you for answering.
It looks like this
2017-04-03 23:57:46

4

That should be fine. So is the date filter failing (resulting in a _dateparsefailure tag in your events) or is the problem that the resulting field in ES isn't a date field? In the latter case the problem is probably that the field at some point was mapped as a string and that won't change just because what you're currently sending in that field looks like a timestamp. One of way fixing the problem is deleting the index and starting over (since you might not have useful data there anyway), optionally with an index template that explicitly maps the log_timestamp field as a date field.

Any particular reason you want to call the timestamp field log_timestamp field instead of the default @timestamp? The latter will already have the correct mapping if you use Logstash's default index template.

5

Thanks for the answer it helped us solving it by doing the following.

Delete the old index.
Add the following field under the properties field to our filebeat template.

"log_timestamp": {
  "type": "date"
 }

We use @timestamp as well but want an additional date for the log.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.