Timezone parameter has no impact in Date filter plugin

Elastic Stack Logstash
1

TLDR: my timestamp value is already in UTC, I am trying to prevent elastic from converting it to UTC again.

I am using the JDBC input in logstash to query a SQL server database. When I look at a specific record in SQL Server Management Studio the date is "2021-07-08 15:35:09.057".

Here is my logstash config file

input {
  jdbc {
    ...
    statement => "SELECT 
        [TimeStamp] as parsethis
        ...
  }
}

filter {
  mutate {
    convert => {
      "parsethis" => "string"
    }
  }
  date {
    match => ["parsethis", "ISO8601"]
    target => "parsedUTC"
    timezone => "UTC"
  }
  date {
    match => ["parsethis", "ISO8601"]
    target => "parsedEST"
    timezone => "EST5EDT"
  }
}

In Dev Tools, here is the record:

        "_source" : {
          "parsethis" : "2021-07-08T19:35:09.057Z",
          "parsedEST" : "2021-07-08T19:35:09.057Z",
          "parsedUTC" : "2021-07-08T19:35:09.057Z"
        }

Notice how the parsedEST and parsedUTC fields above have the exact same value. What I would expect (and what I need) is for one of the values to be 2021-07-08 15:35:09.057 which is the original value in the SQL database.

I understand that elasticsearch wants to store date values in UTC. I don't understand how I am supposed to tell elasticsearch that the value is already in UTC and not to convert it to that again.

Why does this method not work? What is the correct way to address this?

2

I think I have solved the problem I was having.

In the JDBC input I added this line: jdbc_default_timezone => "UTC"

That converted all the time values to 15:...

I still don't understand the point of the timezone parameter in the data filter plugin since it is seemingly has no effect.

3

It has no effect because the timezone is specified in the date string itself. The Z at the end of the strings mean they are in Zulu, or UTC timezone. You cannot override that with the timezone option.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.