Combining two events in one to calculate time difference

Elastic Stack Logstash
1

Hello team,
I have case where information is being displayed in different lines. These lines are not even consecutive lines. Unique field is log_processed.companyId. I have to display time difference between entry and exit log.

I have tried with aggregate filter but it didn't work

Sample log:

{"timestamp":"2021-11-17T05:52:59:613","level":"info","message":"ENTRY: INTERNAL_HTTP_REQUEST","userId":"rahul","companyId":"abc"}

{"timestamp":"2021-11-17T05:52:59:768","level":"info","message":"EXIT: INTERNAL_HTTP_REQUEST","userId":"rahul","companyId":"abc"}

Aggregate filter:

if [log_processed.message] == "ENTRY: INTERNAL_HTTP_REQUEST" {
        aggregate {
            task_id => "%{log_processed.companyId}"
            code => "map['started'] = event['@timestamp']"
            map_action => "create"
        }
    }

    if [log_processed.message] == "EXIT: INTERNAL_HTTP_REQUEST" {
        aggregate {
            task_id => "%{log_processed.companyId}"
            code => "event['duration'] = event['@timestamp'] - map['started']"
            map_action => "update"
            push_map_as_event_on_timeout => true
            timeout_task_id_field => "log_processed.companyId"
            timeout => 60 # 1 minutes timeout
			timeout_tags => ['_aggregatetimeout']
        }
    }
2

Hello @Badger ,
Can you please help me here

3

Hi @mangeshmj1992 ,

You should take a look at the Elapsed Filter as from your description that does what you're looking for.

We have 'start' and 'end' events (separate documents being fed by Logstash into Elasticsearch). They both contain a unique ID so it's possible to correlate which 'end' event relates to which 'start' event.
The elapsed plugin can monitor for these events, correlate them together and provide the 'elapsed time' by subtracting the start event timestamp from the end event timestamp.

Note that it's critical to have a unique correlation ID that both events contain.

Hope that helps,
Steve

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.