# Aggregate filter - stop after N events

**URL:** https://discuss.elastic.co/t/aggregate-filter-stop-after-n-events/210950
**Category:** Logstash
**Created:** [December 7, 2019, 4:13am UTC](https://discuss.elastic.co/t/aggregate-filter-stop-after-n-events/210950 "2019-12-07T04:13:36Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![tomr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tomr/32/48260_2.png) [@tomr](https://discuss.elastic.co/u/tomr)
#### Post date: [December 7, 2019, 4:13am UTC](https://discuss.elastic.co/t/aggregate-filter-stop-after-n-events/210950/1 "2019-12-07T04:13:36Z")

</div>

Short version: can the aggregate filter be made to stop after it receives 2 events for a given task ID?

I'm using the `aggregate` filter to successfully combine two very similar apache log lines, one of which contains a valid auth name and the other a duration (don't ask!)

The events arrive up to a minute apart, and while I'm loathe to miss any I have two problems with setting a large timeout. Firstly, I worry about having huge numbers of in-flight events and maps clogging up the aggregate filter and its single pipeline worker. Secondly, I'd rather not wait longer than I have to to see the events.

Ideally I'd like to set a timeout of around 300s, but have the map pushed immediately once two events have arrived. Is this possible?

In case it's of interest to anyone who stumbles across this post, my approach is to create a murmur3 fingerprint of all fields except `duration` and `auth`, then use that as `task_id`.

```auto
fingerprint {
  source => ["all", "common", "fields"]
  concatenate_sources => true
  method => "MURMUR3"
  target => "[@metadata][aggregate_id]"
}
aggregate {
  task_id => "[@metadata][aggregate_id]"
  code => "
    map['all'] ||= event.get('all')
    map['common'] ||= event.get('common')
    map['fields'] ||= event.get('fields')
    map['duration'] ||= event.get('duration')
    map['auth'] ||= event.get('auth')
    event.cancel()
  "
  push_map_as_event_on_timeout => true
  timeout => 90
}

```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 4, 2020, 4:13am UTC](https://discuss.elastic.co/t/aggregate-filter-stop-after-n-events/210950/2 "2020-01-04T04:13:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
