Field Timezone conversion

Elastic Stack Elasticsearch
1

I have data in Singapore and I want to save data as is and display as is.
I understand elk everything gets to UTC. hence when I save data to elasticsearch by default it moves to UTC and gets messed.
reason I said messed up because ( when someone runs rest query it does not shows up where it occur)

if I use timezone=>"something" it saves data as is.
i.e 10:10:10 will saves as 10:10:10 (when I look actual data in discovery or run sql found data where it occur)

problem now I have is that "NOW" time in kibana is eight hour behind. ( because I use timezone=>"Etc/GMT" on that data.)

I have been trying to figure out this from more then month now.

2

Example 1

input {
   generator {
      message => '{"submitted_date": "2020-01-14 10:05:11.11"}'
      count => 1
   }
}

filter {
   json { source => "message" }
   date { match => ["submitted_date", "yyyy-MM-dd HH:mm:ss.SS"]
        timezone => "Australia/Perth"
        target => "submitted_date"
   }
   mutate { remove_field => ["message","host","path"] }

Output looks like this

{
    "submitted_date" => 2020-01-14T02:05:11.110Z,  (which is 8 hour early then what My actual data is)
}

And that is not what I want.

now I use timezone => "Etc/GMT" then output looks like

{
    "submitted_date" => 2020-01-14T10:05:11.110Z,
}

And hence when some one runs SQL against this data it shows up at correct timeframe.

Now when I use kibana using "Etc/GMT" as time setting this data show up at same timeframe. But NOW is shifted few hours.

as you can see actual time on system is 12:46am. but NOW time on graph is 16:46
(which is 8 hour behind). how do I get this to show up 12:46

image

3

If I let Elasticsearch and kibana handle all the timestuff then Kibana shows up correctly
but data inside elasticsearch saved with some time offset. and hence sql query comes result comes out wrong.

4

Anyone, any idea? any direction will be welcome

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.