Convert NanoSecond Unix timestamp

Elastic Stack Logstash
1

Hi Folks,

Any idea how to convert the nanosecond unix timestamp in logstash filter?

date {
                match => [ "eventtime","UNIX_MS", "ISO8601" ]
                target => "Epoch"
                timezone => "UTC"
                }

Apparently its not working

 "Epoch" => +53023221-08-28T18:34:21.798Z,
2

Welcome to the community Jay.
Can you explain what are first 8 digits?
"Epoch" => +53023221-08-28T18:34:21.798Z

Or show the eventtime field value.

3

Thanks @Rios.

This is the another example of the eventtime field value:

eventtime: 1673188498490039856
Target "Epoch": +53023202-10-20T09:53:59.856Z

I understand that UNIX_MS (milisecond) plugin will not work on nanoseconds timestamp. Any work around?

4

It is 53 million years in the future.

The simplest solution is to throw away all the sub-millisecond precision.

    mutate { add_field => { "eventtime" => 1675199470000000000 } }
    mutate { gsub => [ "eventtime", "\d{6}$", "" ] }
    date { match => [ "eventtime","UNIX_MS", "ISO8601" ] target => "Epoch" timezone => "UTC" }

gives you

     "Epoch" => 2023-01-31T21:11:10.000Z
5

Proof how LS is advance :slight_smile:
Use the simplest solution. If nanosec is mandatory in ES, then use this or this

6

If you need nanosecond precision it can be done in logstash

    mutate { add_field => { "eventtime" => 1675199470123456789 } }
    ruby {
        code => '
            n = event.get("eventtime")
            if n
                match = /(\d+)(\d{9})/.match(n)
                event.set("@timestamp", LogStash::Timestamp.new(Time.at(match[1].to_i, match[2].to_i, :nsec)))
            end
        '
    }

will produce

"@timestamp" => 2023-01-31T21:11:10.123456789Z,
7

Hi @Badger ,

Sorry im not an expert when its comes to logstash filtering. Your solution seems to work but logstash unable to parse the date. Did i miss anything here?

image

image

8

Your eventtime is an array, so you would need to use [eventtime][0] in the date filter. But if it is an array then you never should have gotten a far future date, so I am not sure what your data really looks like.

9

Awesome, its work. Thank you so much for the solution. You just solved the issue that I was struggling with for hours