Calculate time difference between two logs with unique ID

[michaeleino]

hello all,
I'm in the need to calculate the time difference manually without the elapsed plug-in as my end time varies & may got to 24hour with a very big amount of logs. I managed to get the start time from the old log by:

elasticsearch {
                query => "Event:'Sent' AND ID:%{[ID]}"
                index => "mylog*"
                result_size => "1"
                enable_sort => "false"
                fields => { "@timestamp" => "SentTime" }
        }

It actually works fine & return the right date in the format "2017-03-29T22:00:03.000Z"

But it's going bad with: return values in this format "January 18th 1970, 08:07:09.056"

ruby {
          code => "event.set('[SecondsToDeliver]', event.get('@timestamp').to_f - event.get('SentTime').to_f)"
          add_tag => [ "rubyfilter" ]
         }

I'm sure it is a trivial syntax error but i can't manage to catch it to work with logstash V5.

[michaeleino]

this issue is commonly asked in logstash forums, it is really confusing.
Where is the community help!

I wasted here a lot of time & to save others time, the following syntax works fine with logstash 5.3

elasticsearch {
                        hosts => ["Your elastic host"]
                        query => 'Event:"Sent" AND ID:"%{ID}"'
                        fields => { "@timestamp" => "SentTime" }
                        tag_on_failure => [ "NoSent_ID" ]
                }

             date {
                match => ["[SentTime]", "ISO8601"]
                target => "[SentTime]"
             }
        ruby {
                    init => "require 'time'"
                    code => "duration = (event.get('@timestamp') - event.get('SentTime')) rescue nil; event.set('Log_duration', duration); "
                 }