Hi Folks,
Any idea how to convert the nanosecond unix timestamp in logstash filter?
date {
match => [ "eventtime","UNIX_MS", "ISO8601" ]
target => "Epoch"
timezone => "UTC"
}
Apparently its not working
"Epoch" => +53023221-08-28T18:34:21.798Z,
Rios
(Rios)
January 31, 2023, 8:43pm
2
Welcome to the community Jay.
Can you explain what are first 8 digits?
"Epoch" => +53023221 -08-28T18:34:21.798Z
Or show the eventtime field value.
Thanks @Rios .
This is the another example of the eventtime field value:
eventtime: 1673188498490039856
Target "Epoch": +53023202-10-20T09:53:59.856Z
I understand that UNIX_MS (milisecond) plugin will not work on nanoseconds timestamp. Any work around?
Badger
January 31, 2023, 9:18pm
4
It is 53 million years in the future.
The simplest solution is to throw away all the sub-millisecond precision.
mutate { add_field => { "eventtime" => 1675199470000000000 } }
mutate { gsub => [ "eventtime", "\d{6}$", "" ] }
date { match => [ "eventtime","UNIX_MS", "ISO8601" ] target => "Epoch" timezone => "UTC" }
gives you
"Epoch" => 2023-01-31T21:11:10.000Z
2 Likes
Rios
(Rios)
January 31, 2023, 9:25pm
5
Proof how LS is advance
Use the simplest solution. If nanosec is mandatory in ES, then use this or this
1 Like
Badger
January 31, 2023, 9:37pm
6
If you need nanosecond precision it can be done in logstash
mutate { add_field => { "eventtime" => 1675199470123456789 } }
ruby {
code => '
n = event.get("eventtime")
if n
match = /(\d+)(\d{9})/.match(n)
event.set("@timestamp", LogStash::Timestamp.new(Time.at(match[1].to_i, match[2].to_i, :nsec)))
end
'
}
will produce
"@timestamp" => 2023-01-31T21:11:10.123456789Z,
Hi @Badger ,
Sorry im not an expert when its comes to logstash filtering. Your solution seems to work but logstash unable to parse the date. Did i miss anything here?
Badger
January 31, 2023, 9:42pm
8
Your eventtime is an array, so you would need to use [eventtime][0] in the date filter. But if it is an array then you never should have gotten a far future date, so I am not sure what your data really looks like.
1 Like
Awesome, its work. Thank you so much for the solution. You just solved the issue that I was struggling with for hours
system
(system)
Closed
February 28, 2023, 9:57pm
10
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.