# Aggregate filter timeouts and input completion **URL:** \ **Category:** Logstash\ **Created:** [June 11, 2021, 8:24pm UTC](https://discuss.elastic.co/t/aggregate-filter-timeouts-and-input-completion/275719 "2021-06-11T20:24:46Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![jratliff](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jratliff/32/52746_2.png) [@jratliff](https://discuss.elastic.co/u/jratliff)\ **Post date:** [June 11, 2021, 8:24pm UTC](https://discuss.elastic.co/t/aggregate-filter-timeouts-and-input-completion/275719/1 "2021-06-11T20:24:46Z")
I am using logstash to read events from elasticsearch and the aggregate filter to condense them. Because the data I am condensing has no clear start/stop events, we are using the timeout and inactivity\_timeout settings of the aggregate filter and pushing the map as an event. However, we are not getting all of the aggregations. It seems like logstash is exiting as soon as the Elasticsearch input has sent all of the events. I think this is because we use event.cancel() at the end of the aggregation plugin because we do not want the original events. If I use an hour timeout for the hours worth of data I'm reading from elastic, I never get any results. If I reduce the timeout, I can get partial results, but I don't think I am getting all of them. Is logstash exiting because no more events exist and not allowing the aggregate filter to timeout and push new events? Is there anything I can do about this? Can I tell the aggregate filter to push all the events when the input is complete? Or can I delay the exit until the timeout value? Thanks. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [June 11, 2021, 8:42pm UTC](https://discuss.elastic.co/t/aggregate-filter-timeouts-and-input-completion/275719/2 "2021-06-11T20:42:39Z")
> [@jratliff](#): > > Can I tell the aggregate filter to push all the events when the input is complete? The filter will flush any events still in the map when logstash shuts down. ---
**Author:** ![jratliff](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jratliff/32/52746_2.png) [@jratliff](https://discuss.elastic.co/u/jratliff)\ **Post date:** [June 14, 2021, 3:24pm UTC](https://discuss.elastic.co/t/aggregate-filter-timeouts-and-input-completion/275719/3 "2021-06-14T15:24:09Z")
I want to be sure we are talking about the same thing. I believe the filter has processed all the events from the input. Now we have only maps that have not had a timeout or end event. Are you saying that upon logstash exit, the aggregate filter is supposed to push all the maps as new events, just as if it had reached the timeout value? If I am understanding you, then I do not believe this is happening. When I have a high timeout value, and am only using the stdout output plugin, I get NO output. I see no maps that were pushed as events. I only see output at all when I lower the timeout, and I do not believe it has pushed all the maps even then. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [June 14, 2021, 4:42pm UTC](https://discuss.elastic.co/t/aggregate-filter-timeouts-and-input-completion/275719/4 "2021-06-14T16:42:55Z")
My bad. The final flush (when logstash exits) is called but does not do anything when the push\_map\_as\_event\_on\_timeout is used. It only flushes events when push\_previous\_map\_as\_event is used. Periodic flushes are expected to work for both. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 12, 2021, 4:43pm UTC](https://discuss.elastic.co/t/aggregate-filter-timeouts-and-input-completion/275719/5 "2021-07-12T16:43:05Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.