Set date time transformations from date filter

Elastic Stack Logstash
1

Here is a (partial) syslog entry (ignore IP validity they have been scrambled):

Jun 15 00:51:19 139.133.7.190 (squid-1): src="172.16.0.51" src_port="49530" dst="122.355.89.1" dst_port="80" local_time="15/Jun/2015:00:51:19 +0300"`

Here is the way I am using date filter

  date {
    # target => "@timestamp"
    match => [ "local_time", "dd/MMM/yyyy:HH:mm:ss +0300" ]
    # local_time="21/Jun/2015:23:45:39 +0300"
    tag_on_failure => ["no_date_match"]
    timezone => "Europe/Athens"
  }

Can anyone explain why my document gets a @timestamp shifted by 3 hours?, i.e. 2015-06-14T21:51:19.000Z

I just want to have in my @timestamp the exact value (without the +0300 part of course) appearing in the local_time field!

2

The Elastic stack always stores times as UTC. You have specified a timezone of Europe/Athens, which is three hours ahead of UTC.

If you are using Kibana that will typically adjust things back to the browser's local timezone.

3

Removing the

    timezone => "Europe/Athens"

makes the document having correct @timestamp (which can be viewed via its json representation) but now kibana is the misleading one (3 hours ahead)

4

The Elastic stack stores times as UTC, regardless of which time zone you are in. If your logs have timestamps that are in Europe/Athens you should expect them to be three hours behind in elasticsearch.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.