Dreded date with timezone :)

Elastic Stack Logstash
1

so i m trying to parse XML and ingest . Here is my XML

ReceivedTimestamp and DeliveredTimestamp are in CST (Chicago)

<?xml version="1.0" encoding="UTF-8" standalone="no"?> 2019-12-12T02:10:032019-12-12T02:20:03

Here is my logstash config.

input { stdin { } }
filter {
         xml {
           source => "message"
           store_xml => "false"
           xpath => [
                  "/TransactionDB/TDB_ReceivedTimestamp/text()","ReceivedTimestamp",
                  "/TransactionDB/TDB_DeliveredTimestamp/text()","DeliveredTimestamp"
                  ]
           } # end of xml


        mutate {
        replace => [
              "ReceivedTimestamp" , "%{[ReceivedTimestamp][0]}",
              "DeliveredTimestamp" , "%{[DeliveredTimestamp][0]}"
              ]
         }
          mutate {
        remove_field => [ "message","host"]
    }



         date{
            #timezone => "Etc/GMT+6"
            match => ["ReceivedTimestamp","ISO8601"]
            timezone => "America/Chicago"
    }

        date{
            #timezone => "Etc/GMT+6"
            match => ["DeliveredTimestamp","ISO8601"]
            timezone => "America/Chicago"
    }


}
output {

         stdout { codec => rubydebug {}}
  }

Here is my output.

{
"DeliveredTimestamp" => "2019-12-12T02:20:03",
"@version" => "1",
"@timestamp" => 2019-12-12T08:20:03.000Z,
"ReceivedTimestamp" => "2019-12-12T02:10:03"
}
[2019-12-12T14:10:34,632][INFO ][logstash.runner ] Logstash shut down.

so the thing presentation layer (kibana) is converting ReceivedTimestamp and DeliveredTimestamp with -6 hours. so that shows up as of Dec 11, 2019 @ 20:10:03.000 and Dec 11, 2019 @ 20:20:03.000 which is wrong.

how do i convert those to UTC + 6 or CST or correct time stamp so it reflects right?

2

The first date filter sets @timestamp to 2019-12-12T08:10:03.000Z, the second overwrites it with 2019-12-12T08:20:03.000Z. Perhsp you should use the target option to tell the date filter where to store the result.

3

That was it :slight_smile: Thanks a lot for quick response and solution.

Raj

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.