Timestamp is ingest timestamp, not event timestamp

The issue is that the field already exists. The code does different things depending on the type of the existing field. If the existing field is a LogStash::Timestamp then it logs a DEBUG message saying it will leave it unmodified.

[2020-09-11T13:50:33,173][DEBUG][logstash.filters.grok    ][main][91fb85f2e5abc66c776cd383ec1ba5840bbc036e454a5930ae9c1db8f095cefe] Not adding matched value - found existing (LogStash::Timestamp) {:field=>"@timestamp", :value=>"2020-08-20 20:10:27"}

Now you might think that you could get around this by adding

overwrite => [ "@timestamp" ]

to your grok filter, but that does not work. logstash knows that @timestamp should be a LogStash::Timestamp and when grok tries to event.set it to a string it gets a WARN:

Grok regexp threw exception {:exception=>"wrong argument type String (expected LogStash::Timestamp)"

Change your grok filter to start with '%{TIMESTAMP_ISO8601:[@metadata][timestamp]} and parse that using a date filter.

date { match => [ "[@metadata][timestamp]", "YYYY-MM-dd HH:mm:ss" ] }
1 Like