# How to use aggregate function in logstash to calculate difference between 'elapsed' time fields?

**URL:** <https://discuss.elastic.co/t/how-to-use-aggregate-function-in-logstash-to-calculate-difference-between-elapsed-time-fields/230004>\
**Category:** Logstash\
**Created:** [April 27, 2020, 6:14pm UTC](https://discuss.elastic.co/t/how-to-use-aggregate-function-in-logstash-to-calculate-difference-between-elapsed-time-fields/230004 "2020-04-27T18:14:03Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nani\_20](https://avatars.discourse-cdn.com/v4/letter/n/4491bb/32.png) [@Nani\_20](https://discuss.elastic.co/u/Nani_20)\
**Post date:** [April 27, 2020, 6:14pm UTC](https://discuss.elastic.co/t/how-to-use-aggregate-function-in-logstash-to-calculate-difference-between-elapsed-time-fields/230004/1 "2020-04-27T18:14:03Z")

</div>

I am using the latest version of logstash(7.6.2). I am trying to calculate the total time spent in system excluding time spent between Out2 and In2 i.e. **[(Out3-In1) - (Out2-In2)]** for a given ID. Find below my sample data with columns Timestamp, Event, and ID

**Sample Data**

```auto
18/May/2015:02:05:10 +0000 In1 100
18/May/2015:02:05:15 +0000 Out1 100
18/May/2015:02:05:26 +0000 In2 100
18/May/2015:02:05:48 +0000 Out2 100
18/May/2015:15:05:08 +0000 In3 100
18/May/2015:16:05:03 +0000 Out3 100

```

I am currently using two elapsed functions and 2 aggregate functions as shown below in the config file

**Config File**

```auto
filter{
    grok{
        match => {"message" => "%{HTTPDATE:timestamp} %{WORD:Event} %{NUMBER:ID}"}
        add_tag => ["%{Event}"]
    }
    date{
        match => ["timestamp", "dd/MMM/yyyy:HH:mm:ss Z"]
    }
    elapsed {
        start_tag => "In1"
        end_tag => "Out3"
        unique_id_field => "ID"
    }
    elapsed {
        start_tag => "In2"
        end_tag => "Out2"
        unique_id_field => "ID"
    }
    if [Event] == "Out2" {
        aggregate {
            task_id => "%{ID}"
            code => "map['time1'] = (event.get('elapsed_time')*1000).to_i"
            map_action => "create"
        }
    }
    if [Event] == "Out3" {
        aggregate {
            task_id => "%{ID}"
            code => "map['time2'] = (event.get('elapsed_time')*1000).to_i - map['time1']; event.set('time1', map['time2'])"
            map_action => "update"
            end_of_task => true
        }
    }
}

```

I was able to generate elapsed time fields but was unable to calculate the difference between then using Agreggate filter. Find below my output

Is there anything I am missing?

Any help is appreciated! Thanks in advance

---

<div class="post-metadata">

**Author:** ![Nani\_20](https://avatars.discourse-cdn.com/v4/letter/n/4491bb/32.png) [@Nani\_20](https://discuss.elastic.co/u/Nani_20)\
**Post date:** [April 28, 2020, 6:49am UTC](https://discuss.elastic.co/t/how-to-use-aggregate-function-in-logstash-to-calculate-difference-between-elapsed-time-fields/230004/2 "2020-04-28T06:49:39Z")

</div>

I have added `piepline.workers:1` in the logstash.yml and it works now.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 28, 2020, 4:41pm UTC](https://discuss.elastic.co/t/how-to-use-aggregate-function-in-logstash-to-calculate-difference-between-elapsed-time-fields/230004/3 "2020-04-28T16:41:15Z")

</div>

You may also need to [disable java\_execution](https://github.com/elastic/logstash/issues/10938) until you get to 7.7.

---

<div class="post-metadata">

**Author:** ![Nani\_20](https://avatars.discourse-cdn.com/v4/letter/n/4491bb/32.png) [@Nani\_20](https://discuss.elastic.co/u/Nani_20)\
**Post date:** [April 28, 2020, 6:55pm UTC](https://discuss.elastic.co/t/how-to-use-aggregate-function-in-logstash-to-calculate-difference-between-elapsed-time-fields/230004/4 "2020-04-28T18:55:32Z")

</div>

@Badger Thanks for letting me know. But, where do I need to disable this? In the `logstash.yml` file?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 28, 2020, 7:16pm UTC](https://discuss.elastic.co/t/how-to-use-aggregate-function-in-logstash-to-calculate-difference-between-elapsed-time-fields/230004/5 "2020-04-28T19:16:03Z")

</div>

You can do it in logstash.yml or on the command line.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 26, 2020, 7:16pm UTC](https://discuss.elastic.co/t/how-to-use-aggregate-function-in-logstash-to-calculate-difference-between-elapsed-time-fields/230004/6 "2020-05-26T19:16:04Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.