Convert timestamps to UTC(UNIX) from PST,IST

Elastic Stack Logstash
1

Hello,

How can I convert my timestamps in UNIX from different timezones such as EDT, PST/IST to UTC in Logstash filters?

2

With date or ruby filter. If you need help, you have to show an example of what you are trying to do.

3

This block of code I use is to convert timestamp to UNIX and

event.set('timestamp', event.get('timestamp_object').to_i * 1000)

we use this block to convert the timestamp to UTC
date {
timezone => "UTC"
match => [ "timestamp_local" , "yyyyMMddHHmmss" ]
target => "timestamp_object"
}

Ex:

 "timestamp_local" => "20190730112353"
"timestamp_object" => 2019-07-30T11:23:53.000Z"
 "timestamp" => 1564485833000"

I can see the code is converting into a different format rather than a timezone

4

Huh?

So first you have a “timestamp_object” field, which contains timestamp in Unix timestamp format. You multiply it by 1000 to get it into milliseconds and store the result into “timestamp” field.
Then from somewhere comes the timestamp_local field, which uses “yyyyMMddHHmmss” format, you parse the timestamp from that field and overwrite “timestamp_object” field with the result.

I’m not sure what is the issue, what do you expect to be different?

5

I apologize if I didn't put the question correctly. my input is "timestamp_local" which needs to be converted to UTC UNIX format.
so for that, we initially tried to convert it to UTC which is the below code
date {
timezone => "UTC"
match => [ "timestamp_local" , "yyyyMMddHHmmss" ]
target => "timestamp_object"
}

Now, I am expecting the timestamp_object to be in UTC format but I don't see the timestamp_object in UTC and then comes the conversion to UNIX in ms.

event.set('timestamp', event.get('timestamp_object').to_i * 1000)

which is working fine..so my only issue is the block which needs to convert to UTC timezone is not actually converting it..It looks like it's only changing the format not the timezone

6

A date filter always converts to UTC. The timezone option is used to tell what timezone it is converting from, i.e. the name of your local timezone. Note that PST and IST are ambiguous. Use whatever zone name the Joda documentation lists for the same offset.

7

Oh cool..so when i specify timezone as UTC ,all i am doing is specifying that the server timestamp is in UTC timezone and not the timezone that i need it to be converted to? So If i specify timezone as Etc/GMT+4 for example...it is going to convert timestamp "timestamp_local" to Etc/GMT+4 ?

8

If you timestamp is in Etc/GMT+4 then you should specify that as the value of the timezone option. As I said, it will get converted to UTC.

9

Cool. Thank You !

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.