Convert timezone of @timestamp field

Elastic Stack Logstash
1

I need to do this because we are putting all out index with Etc/UTC timezone
and setting up all user on kibana with Etc/UTC timezone.

that way when use login to kibana they see data on proper time
when we run sql query against elasticsearch we get data on proper time.

now some of the index are using @timestamp and they are on +6 timezone

that means

"@timestamp" => 2019-12-20T19:50:01.362Z

if actually 2019-12-20 1:50:01.36 and I want to save it as.

How do I change @timestamp's timezone from what ever default it does to what I want

2

How is @timestamp getting set?

3

@timestamp is getting set default

not setting with any field. what ever default when logstash ran.

4

OK, so mutate+copy it to another field, mutate+convert it to a string, then use a date filter with a timezone option to overwrite @timestamp.

5

something like this ?

mutate { copy => { "@timestamp" => "new_timestamp" } }

mutate { convert => { "new_timestamp" => "string" } }

date { match => ["new_timestamp", "yyyy-MM-dd'T'HH:mm:ss.SSSZ", "ISO8601" ]
timezone => "Etc/UTC"
target => "new_timestamp"
}

did't work

{
"username" => "ROOT",
"@version" => "1",
"new_timestamp" => 2019-12-23T14:58:05.525Z,
"@timestamp" => 2019-12-23T14:58:05.525Z
}

They are identical. new_timestamp should be 08:58:05 (on UTC)

6

The trailing Z says the date is UTC, so the timezone option is ignored. Use mutate+gsub to remove it.

mutate { gsub => [ "new_timestamp", "Z$", "" ] }
1 Like
7

still coming up same.
I don't understand. because I am creating new_timestamp=@timestamp which is already 6+ hour ahead.
My goal is to move @timestamp to 6 hour behind right? so it line up with my timezone.

mutate { copy => { "@timestamp" => "new_timestamp" } }

mutate { convert => { "new_timestamp" => "string" } }
mutate { gsub => ["new_timestamp", "Z$", ""] }

Output looks
{
"@timestamp" => 2019-12-26T14:57:36.100Z,
"new_timestamp" => "2019-12-26T14:57:36.100",
}

Now if I convert this with timezone. it is gone a stay same but will Z at the end.

1 Like
8

@Badger

I have figure out how to deal with this timezone thing.

basically I have data sitting in different timezone. and people from different timezone is gone a check this data which are in different timezone.

for example
user1 - in GMT -- will check data in GMT timezone, will check data in CST timezone and so on..

We want all the data save in ELK as is. i.e if record has time=5:00 CST keep it as is. That way when use from UK login from his own browser he should see that record at 5:00 CST

in order to do that I have put all data with timezone Etc/CST and all user advance Settings - timezone for dataformating = Etc/CST

for UK data Etc/GMT
for Perth data Etc/GMT - timezone and advancesetting and I can see everything as is.

I hope this logic helps someone who are trying to do same

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.