Timezone logstash date filter

Elastic Stack Logstash
1

Hello,
I want to have another date field in Europe/Paris timezone with logstash date filter.

filter {
  mutate {
    add_field =>{
      "my_date" => "%{@timestamp}"
    }
  }

  date {
    match => ["my_date", "yyyy-MM-dd'T'HH:mm:ss.SSSZ"]
    timezone => "Europe/paris"
    target => "my_date"
  }
}

The result is

"my_date" => 2021-08-18T08:36:05.569Z,
"@timestamp" => 2021-08-18T08:36:05.569Z,

What I want is :

"my_date" => 2021-08-18T10:36:05.569Z,
"@timestamp" => 2021-08-18T08:36:05.569Z,

Thank

2

If the timestamp has a Z at the end then it is in UTC and the timezone option is ignored. You can use mutate+gsub to remove it.

3

Thank you for you response.
I use mutate and gsub and extract just "yyyy-MM-dd'T'HH:mm:ss.SSSZ" value
I use the next config

date {
    match => ["my_date", "yyyy-MM-dd'T'HH:mm:ss"]
    timezone => "Europe/paris"
    target => "my_date1"
  }

and the result is

"my_date" => 2021-08-18T06:36:05,
"my_date1" => 2021-08-18T08:36:05.569Z,

different to what I really want

"my_date1" => 2021-08-18T10:36:05,
"my_date" => 2021-08-18T08:36:05.569Z,
4

A date filter always converts a date to UTC. If you tell it the timestamp is in Europe/Paris then it will subtract two hours from the timestamp to do that. I get

   "my_date" => "2021-08-18T06:36:05",
  "my_date1" => 2021-08-18T04:36:05.000Z,

If you want to invert the delta, but keep the start/end dates for DST then this might help

    mutate { add_field => { "[my_date]" => "2021-08-18T06:36:05" } }
    date {
        match => ["my_date", "yyyy-MM-dd'T'HH:mm:ss"]
        timezone => "UTC"
        target => "my_date1"
    }
    date {
        match => ["my_date", "yyyy-MM-dd'T'HH:mm:ss"]
        timezone => "Europe/Paris"
        target => "my_date2"
    }
    ruby {
        code => '
            t1 = event.get("my_date1").to_f
            t2 = event.get("my_date2").to_f

            if t1 and t2
                offset = t1 - t2
                event.set("otherTimestamp", LogStash::Timestamp.new(Time.at(t1 + offset)))
            end
        '
    }

which produces

      "my_date2" => 2021-08-18T04:36:05.000Z,
       "my_date" => "2021-08-18T06:36:05",
      "my_date1" => 2021-08-18T06:36:05.000Z,
"otherTimestamp" => 2021-08-18T08:36:05.000Z
5

Thank you @Badger ,
It works perfectly

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.