Convert datetime to another timezone in logstash

Elastic Stack Logstash
1

I am getting logs from a firewall which are in GMT timezone. For example firewall sending rt=Jun 16 2023 11:24:40 GMT I want to convert that time to MYT rt=June 16 2023 19:24:40 MYT.

How can I do that.

filter {
grok{
match => {"message" => "%{TIMESTAMP_ISO8601:Timestamp} %{SYSLOGHOST:DeviceName} CEF: %{INT:CEF_Version}\|(?<DeviceVendor>\w.*)\|(?<DeviceProduct>\w+-\w+)\|(?<DeviceVersion>\d+\.\d+\.\d+)\|(?<LogSubType>\b\w+(?:-\w+)?\b)\|%{WORD:LogType}\|%{INT:Severity}\|rt=(?<ReceiptTime>%{MONTH} %{MONTHDAY} %{YEAR} %{TIME} %{WORD}) %{GREEDYDATA:extensions}"}
ecs_compatibility => disabled
}}

Here the field holding GMT time is "ReceiptTime"

2

You can't, all datetime fields in Logstash and Elasticsearch are in UTC and this cannot be changed.

Kibana will convert the UTC dates to your current timezone.

3

I am not sending data to kibana or elasticsearch. I am sending them to seperate destination. In some other post i observed Ruby code filter can change the timezone and time value ?

4

Yeah, you would probably need to write a ruby code using the ruby filter to convert it them, Logstash date filter can only convert into UTC, not to other time zones.

Not sure what the code would like, but there are some examples in the forum.

5

This forum post is probably similar to what you want.

6

No. It's not

7

Well, then you will need to try to create a ruby code using the ruby filter to achieve what you want.

8

I found the solution

filter {
  date {
    match => ["timestamp1", "MMM dd yyyy HH:mm:ss z"]
    target => "converted_date"
    timezone => "GMT"
  }

  ruby {
    code => "
      event.set('converted_date', event.get('converted_date').time.localtime('+08:00').strftime('%b %d %Y %H:%M:%S MYT'))
    "
  }
}

this code helps me

9

You can do it in ruby.

    mutate { add_field => { "rt" => "Jun 16 2023 11:24:40 GMT" } }
    date { match => [ "rt", "MMM dd YYYY HH:mm:ss ZZZ" ] }

    ruby {
        code => '
            t = event.get("[@timestamp]").to_f
            t = Time.at(t, in: "+08:00")
            event.set("localtime", t.strftime("%b %d %Y %H:%M:%S MYT"))
        '
    }

will produce

 "localtime" => "Jun 16 2023 19:24:40 MYT",
"@timestamp" => 2023-06-16T11:24:40.000Z,

I don't think core Ruby has enough timezone support to avoid specifying both the offset and the name of MYT.

1 Like
10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.