# Aggregate plugin sends all events to ES **URL:** \ **Category:** Logstash\ **Created:** [April 10, 2020, 6:38am UTC](https://discuss.elastic.co/t/aggregate-plugin-sends-all-events-to-es/227445 "2020-04-10T06:38:01Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![xyz2](https://avatars.discourse-cdn.com/v4/letter/x/e480ec/32.png) [@xyz2](https://discuss.elastic.co/u/xyz2)\ **Post date:** [April 10, 2020, 6:38am UTC](https://discuss.elastic.co/t/aggregate-plugin-sends-all-events-to-es/227445/1 "2020-04-10T06:38:01Z")
Hi , Need to use multiple events and build single aggregated event based on session id to send to ES Here is how data looks like: session open for user abc (session id :1) session open for user bbc (session id :2) ....some operation by user abc (session id :1) ....some openration by user bbc (session id :2) session closed for user abc (session id :1) ...some operation by user bbc (session id :2) session closed for user bbc (session id :2) Am using aggregate filter. Also specified event.cancel() for non final events. Know the start and end events . Have set pipeline workers as 1 However, **all the events are sent to ES**. Checked this post ([[AGGREGATE] - aggregate is working but single rows are sent to elastic too](https://discuss.elastic.co/t/aggregate-aggregate-is-working-but-single-rows-are-sent-to-elastic-too/168880)) and looks like it should work though. Noticed in one of the other posts ([Aggregate - concatenate events](https://discuss.elastic.co/t/aggregate-concatenate-events/126808/2)), it suggests to use multiline in filebeat as well (along with aggregate plugin). Is FB multiline setting really needed if i have to avoid all events to be sent to ES ?? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 10, 2020, 1:55pm UTC](https://discuss.elastic.co/t/aggregate-plugin-sends-all-events-to-es/227445/2 "2020-04-10T13:55:57Z")
What does your filter configuration look like? ---
**Author:** ![xyz2](https://avatars.discourse-cdn.com/v4/letter/x/e480ec/32.png) [@xyz2](https://discuss.elastic.co/u/xyz2)\ **Post date:** [April 10, 2020, 7:41pm UTC](https://discuss.elastic.co/t/aggregate-plugin-sends-all-events-to-es/227445/3 "2020-04-10T19:41:08Z")
It has grok plugin to fetch basic field to differentiate start, middle and end events start aggregate plugin with event.cancel(), code, taskid, map settings Another grok on basic field to fetch the specific operations and fill the map Based on operations, multiple aggregate filters with event.cancel() in each and map\_action "update" Then a final aggregate filter for final event based on basic grok parsed field. including below map\_action =\> "update" end\_of\_task =\> true Haven't set any timeout related fields as not needed as of now. ---
**Author:** ![xyz2](https://avatars.discourse-cdn.com/v4/letter/x/e480ec/32.png) [@xyz2](https://discuss.elastic.co/u/xyz2)\ **Post date:** [April 10, 2020, 8:28pm UTC](https://discuss.elastic.co/t/aggregate-plugin-sends-all-events-to-es/227445/4 "2020-04-10T20:28:07Z")
Solved the issue myself 🙂 Moved all the grok plugins up and only the aggregate plugins down (although the condition statements were repeated ) ---
**Author:** ![xyz2](https://avatars.discourse-cdn.com/v4/letter/x/e480ec/32.png) [@xyz2](https://discuss.elastic.co/u/xyz2)\ **Post date:** [April 10, 2020, 9:18pm UTC](https://discuss.elastic.co/t/aggregate-plugin-sends-all-events-to-es/227445/5 "2020-04-10T21:18:39Z")
I see the sequence of events is not maintained. How do we ensure that ? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 10, 2020, 11:12pm UTC](https://discuss.elastic.co/t/aggregate-plugin-sends-all-events-to-es/227445/6 "2020-04-10T23:12:14Z")
> [@xyz2](#): > > I see the sequence of events is not maintained. How do we ensure that ? Set pipeline.workers to 1 and (for now) set java\_execution false. ---
**Author:** ![xyz2](https://avatars.discourse-cdn.com/v4/letter/x/e480ec/32.png) [@xyz2](https://discuss.elastic.co/u/xyz2)\ **Post date:** [April 11, 2020, 4:46am UTC](https://discuss.elastic.co/t/aggregate-plugin-sends-all-events-to-es/227445/7 "2020-04-11T04:46:18Z")
> [@Badger](#): > > java\_execution false Have already set pipeline worker as 1. Where do we set java\_execution false ? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 11, 2020, 2:28pm UTC](https://discuss.elastic.co/t/aggregate-plugin-sends-all-events-to-es/227445/8 "2020-04-11T14:28:17Z")
You can set it on the command line or in logstash.yml ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 9, 2020, 2:34pm UTC](https://discuss.elastic.co/t/aggregate-plugin-sends-all-events-to-es/227445/9 "2020-05-09T14:34:44Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.