# Aggregate filter does not always work

**URL:** <https://discuss.elastic.co/t/aggregate-filter-does-not-always-work/91206>\
**Category:** Logstash\
**Created:** [June 28, 2017, 11:06pm UTC](https://discuss.elastic.co/t/aggregate-filter-does-not-always-work/91206 "2017-06-28T23:06:01Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Shuxin\_Lin](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@Shuxin\_Lin](https://discuss.elastic.co/u/Shuxin_Lin)\
**Post date:** [June 28, 2017, 11:06pm UTC](https://discuss.elastic.co/t/aggregate-filter-does-not-always-work/91206/1 "2017-06-28T23:06:01Z")

</div>

I use aggregate filter. It is super powerful. But it does not work consistently. What I mean is the aggregate filter sometimes failed to aggregate certain logs. I would say in 90% of cases it works. The failure is not reproducible. It can aggregate a group of events but next time it might fail. I would like to know how to improve the success rate. Any helps appreciate.

---

<div class="post-metadata">

**Author:** ![Shuxin\_Lin](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@Shuxin\_Lin](https://discuss.elastic.co/u/Shuxin_Lin)\
**Post date:** [June 28, 2017, 11:12pm UTC](https://discuss.elastic.co/t/aggregate-filter-does-not-always-work/91206/2 "2017-06-28T23:12:25Z")

</div>

I could include my code if anyone is interested.

```auto
      # aggregate: start
      if [logger] == "LOG Start" {
        aggregate {
          task_id => "%{id}"
          code => "
                   map['start_time'] = event.get('@timestamp');
                  "
          map_action => "create"
        }
      }

      # aggregate: update
      if [logger] == "LOG Stop" {
        aggregate {
          task_id => "%{id}"
          code => "
                   event.set('start_time', map['start_time']);
                   map['stop_time'] = event.get('@timestamp');
                   event.set('stop_time', map['stop_time']);
                  "
          add_tag => ["aggregate_success"]
          map_action => "update"
          end_of_task => true
          timeout => 100
        }

        ruby {
          init => "require 'time'"
          code => 'event.set("transfer_time", event.get("stop_time") - event.get("start_time") )'
          add_tag => ["calculated_time_difference"]
        }
      }

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 26, 2017, 11:17pm UTC](https://discuss.elastic.co/t/aggregate-filter-does-not-always-work/91206/3 "2017-07-26T23:17:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.