Logstash date filter performs a timeshift of 3 hours

Elastic Stack Logstash
1

Filtering some syslog logs (which are actually stored in a file as follows):

grok {
  match => { 'message' => '%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}' }
  add_field => [ 'received_at', '%{@timestamp}' ]
  add_field => [ 'received_from', '%{syslog_hostname}' ]
  add_field => [ 'unix_time', '0' ]
}

And then adding timestamp (should match a local_time field):

  date {
    target => "@timestamp"
    match => [ "local_time", "dd/MMM/yyyy:HH:mm:ss Z" ]
    # local_time="21/Jun/2015:23:45:39 +0300"
    locale => "en_us"
    tag_on_failure => ["no_date_match"]
  }

Here is an example local_time field value from my original logs:

local_time="15/Jun/2015:00:51:19 +0300"

However, my documents end up having a timestamp of 3hours behind (i.e. earlier) compared to what is mentioned in local_time

2

elasticsearch stores times as UTC. If your logs are not UTC then you need to supply the timezone option to the date filter to tell it what timezone they are in.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.