No log entry after adding date match filter

With the date fields below commented out, I get events logged, but with the wrong timestamp. Removing the comment I get nothing logged and no error messages, at least that I can find. The timezone line doesn't matter either.

Any ideal?

Thanks

      grok {
            match => ["message", "%{TIMESTAMP_ISO8601:log_timestamp} %{IPORHOST:s-ip} %{WORD:cs-method} %{URIPATH:cs-uri-stem} %{NOTSPACE:cs-uri-query} %{NUMBER:s-port} %{NOTSPACE:cs-username} %{IPORHOST:clientip} %{NOTSPACE:cs-useragent} %{NOTSPACE:referrer} %{NUMBER:sc-status} %{NUMBER:sc-substatus} %{NUMBER:sc-win32-status} %{NUMBER:time-taken}"]
      }

date {

match => [ "log_timestamp", "YYYY-MM-dd HH:mm:ss" ]

timezone => "US/Central"

}

      geoip { source => "clientip"  target => "geoip"}

      mutate {
          replace => { "[type]" => "iis" }
          lowercase => [ "cs-username" ]
          remove_field => [ "cs-method", "cs-uri-query", "s-port", "referrer", "sc-substatus", "sc-win32-status",
                           "time-taken", "message" ]
      }

More research, no solution. With this code in the date block above

      date {
          match => [ "log_timestamp", "YYYY-MM-dd HH:mm:ss" ]
          remove_field => [ "log_timestamp" ]
          target => "log_timestamp_test"
      }

log_timestamp_test has the tiny clock icon and a sample displays as December 21st 2016, 03:11:42.000 and the log_timestamp field is dropped, so I know it's going thru the code :slight_smile:

If I make the target => "@timestamp", I still don't get anything sent to elasticsearch.

It would probably help if you showed an example of an actual event.

I tried to add an example last night but I don't see it. Here is a log entry, with some sanitization:

2016-12-10 23:59:58 10.10.10.1 POST /EWS/Exchange.asmx &CorrelationID=;&cafeReqId=f7e635f8-e9ff-4c24-91f9-7259c4d8e5bc; 80 DOMAIN\user 192.168.0.1 Mac+OS+X/10.12.1+(16B2555)+CalendarAgent/384 - 200 0 0 15

I found that adding timezone => "Etc/UTC" fixed the problem.

     date {
          match => [ "log_timestamp", "YYYY-MM-dd HH:mm:ss" ]
          remove_field => [ "log_timestamp" ]
          target => "@timestamp"
          timezone => "Etc/UTC"
      }

I noticed there isn't a "Z" in the timestamp in the this log.

I'm new in ELK, we're just seeing what we can develop.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.