# How to use aggregate filter with multiple workers **URL:** \ **Category:** Logstash\ **Created:** [November 21, 2018, 4:07am UTC](https://discuss.elastic.co/t/how-to-use-aggregate-filter-with-multiple-workers/157621 "2018-11-21T04:07:44Z")\ **Posts on this page:** 1\ **Showing post:** 2
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 21, 2018, 6:54am UTC](https://discuss.elastic.co/t/how-to-use-aggregate-filter-with-multiple-workers/157621/2 "2018-11-21T06:54:44Z")
The aggregate filter indeed has this limitation, which limits performance considerable and prevents scaling to multiple threads and Logstash instances. To get a solution that scales it is probably better to have a solution that does not rely on the ingest layer to handle this. One option could be to have a batch process that periodically queries new data and updates documents where needed. This would typically run externally to Elasticsearch and be implemented using one of the language client. You could also create an [entity-centric index](https://www.youtube.com/watch?v=yBf7oeJKH2Y) where you store a single document per UUID (and use this as the document ID). When you find a document that should be aggregated, you update this document (first time it would be indexed) while at the same time writing the document to the standard index. --- _[View the full topic](https://discuss.elastic.co/t/how-to-use-aggregate-filter-with-multiple-workers/157621)._