Aggregate?

Elastic Stack Logstash
11

Up to me, the right solution for your need is to use 'date' filter and then 'elapsed' filter.

  • date filter allows you to put your message date (ex: 15-04-2016T10:00:00:000+UTC) in @timestamp field.
  • then elapsed filter will compute the elapsed time between start event and end event (using @timestamp field) and will store duration in 'elapsed.time' field in end event.

But you have to know one thing : computed duration is in seconds. If you want a more precise duration (in milliseconds for example), you will have to use aggregate filter.
In all cases, you must first use 'date' filter to set message date in @timestamp field.

Here is the logstash configuration using aggregate filter :

date {
     match => [ "_timestamp", "dd-MM-yyyy'T'HH:mm:ss:SSSZ" ]
}

if [start] {
        aggregate {
             task_id => "%{taskid}"
            map_action => "create"
             code => "map['start_timestamp'] = event['@timestamp']"
        }
}
if [end] {
        aggregate {
             task_id => "%{taskid}"
            map_action => "update"
             code => "event['duration'] = event['@timestamp'] - map['start_timestamp']"
            end_of_task => true
        }
}

Hope it helps.

1 Like