Locale automatically being set in date filter

Elastic Stack Logstash
1

I have a timestamp in my log and its already in the timezone that I want. But when I apply a date filter to it to convert it into a timestamp, it automatically is converted into my local time which I don't want because it already is and therefore it is converted unnecessarily and gives me wrong timestamp. How do I avoid this?

Please help!

2

Can you give an example, including your configuration and information about the timezone of all hosts involved?

3

Yes of course. Below is the config file.

input{
    file{
        path => ["/home/punit/spark.out"]
        start_position => "beginning"    
        codec => multiline 
                    {
                                pattern => "^%{NUMBER}/%{NUMBER}/%{NUMBER}"
                                negate => true
                                what => "previous"
                        }
        }
}
filter{
    grok{
        match => {"message" => "%{NUMBER:log_year}/%{NUMBER:log_month}/%{NUMBER:log_day} %{TIME:log_time} %{LOGLEVEL:log_level} %{DATA:component}: %{GREEDYDATA:log_message}"}
    }
    mutate{
        add_field => {"log_timestamp" => "%{log_year}-%{log_month}-%{log_day} %{log_time}"}
        rename => {"@timestamp" => "event_timestamp"}
                remove_field => ["@version","message","path","log_year","log_month","log_day","log_time"]
    }
    date{
        match => ["log_timestamp","yy-mm-dd HH:mm:ss"]
        target => "log_timestamp"
    }
}
output {
     stdout { codec => json }
    }

Its automatically doing +6:30 to the log timestamp even though I have not specified any timezone or locale.

4

You didn't provide all information that was asked for. Are you taking into account that the date filter always converts timestamps to UTC?

5

No I didn't know that it converted the timestamp to UTC. And what extra info do I have to provide?

6

I want the time as it is in the log file.

7

I said

Can you give an example, including your configuration and information about the timezone of all hosts involved?

and so far I've only seen the configuration. By example I mean what the timestamp looks before the date filter and what it looks like afterwards. The difference you're seeing is probably because of the conversion to UTC.

8

Before date filter:

16/01/27 16:24:47

After date filter:

2016-01-27T10:54:47.000Z

Here is a line from the file:

16/01/27 16:24:47 INFO Master: Registered signal handlers for [TERM, HUP, INT]

9

If your local timezone is UTC+6:30 then this is the expected behavior. The date filter always converts to UTC. This is not configurable.

10

Okay thanks a lot @magnusbaeck. I think I was fretting for no reason.