# Logstash Aggregate filter plugin works in 7.5.0, not in 6.6.1 **URL:** \ **Category:** Logstash\ **Created:** [January 8, 2020, 2:49pm UTC](https://discuss.elastic.co/t/logstash-aggregate-filter-plugin-works-in-7-5-0-not-in-6-6-1/214260 "2020-01-08T14:49:08Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![bjosve](https://avatars.discourse-cdn.com/v4/letter/b/ecae2f/32.png) [@bjosve](https://discuss.elastic.co/u/bjosve)\ **Post date:** [January 8, 2020, 2:49pm UTC](https://discuss.elastic.co/t/logstash-aggregate-filter-plugin-works-in-7-5-0-not-in-6-6-1/214260/1 "2020-01-08T14:49:08Z")
Hi, I have a problem with Logstash Aggregate filter. The exact same filter works in Logstash 7.5.0 but not in Logstash 6.6.1 and I can't find out why. The Aggregate filter version in 7.5.0 is 2.9.1 and in 6.6.1 it is 2.9.0 - and it doesn't seems as it happened so much between these versions. The input, filters and output, ``` input { file { path => "/mnt/testlogs/*" } } # Before cloning: # Generate uuid, in order to match the original and the cloned event. # If there is another field to match the events it can be used as well. filter { ruby { code => "event.set('uuid', rand(36**10).to_s(36))" } } # Cloning the event filter { clone { clones => ["cloned"] } } filter { # Adding a field to the cloned event if [type] == "cloned" { mutate { add_field => { "origin.hostname" => "myhost" } } mutate { copy => { "origin.hostname" => "hfolder" } } # In the cloned event, # Creating an aggregate map that is shared between events that have the same task_id = uuid. # Adding the first and second field to the map aggregate { task_id => "%{uuid}" code => "map['hfolder'] ||= event.get('hfolder')" } # Clean up - remove uuid mutate { remove_field => ["uuid"] } # In the original event # Declare the aggregate map with the same task_id as the cloned event. # Copy the fields from the aggregate map to the event. # } else { aggregate { task_id => "%{uuid}" code => "event.set('hfolder', map['hfolder'])" # Delete the aggregate map from memory, as it is no longer needed. end_of_task => true } # Clean up - remove uuid mutate { remove_field => ["uuid"] } # The mutate filter below is here only for debugging purposes. It can be deleted. mutate { add_field => { "iscloned" => "no" } } } } output { if [type] == "cloned" { stdout { codec => rubydebug } } else { stdout { codec => rubydebug } } } ``` Result in Logstash 6.6.1, ``` "iscloned" => "no", "@version" => "1", "message" => "(deleted)", "path" => "/mnt/testlogs/test.log", "@timestamp" => 2020-01-08T14:02:50.743Z, "hfolder" => nil "@version" => "1", "host" => "216d984a3495", "message" => "(deleted)", "origin.hostname" => "myhost", "type" => "cloned", "path" => "/mnt/testlogs/test.log", "@timestamp" => 2020-01-08T14:02:50.743Z, "hfolder" => "myhost" ``` Result in Logstash 7.5.0, ``` "host" => "9c76c3424d39", "type" => "cloned", "path" => "/mnt/testlogs/test.log", "hfolder" => "myhost", "message" => "(deleted)", "@version" => "1", "origin.hostname" => "myhost", "@timestamp" => 2020-01-08T13:18:00.424Z "@timestamp" => 2020-01-08T13:18:00.380Z, "path" => "/mnt/testlogs/test.log", "hfolder" => "myhost", "message" => "(deleted)", "iscloned" => "no", "@version" => "1", "host" => "9c76c3424d39" ``` pipeline.workers is set to 1 in both environments. In 6.6.1 the original (not clone) event "hfolder" gets "nil" as value. In 7.5.0 "myhost" as expected. I am grateful for all suggestions and input. /Bjorn ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [January 8, 2020, 3:01pm UTC](https://discuss.elastic.co/t/logstash-aggregate-filter-plugin-works-in-7-5-0-not-in-6-6-1/214260/2 "2020-01-08T15:01:48Z")
In 7.5.0, if you set '--pipeline.java\_execution false' on the command line does it stop working? If so, you are relying on java re-ordering your events (which is a bug that will one day get fixed). ---
**Author:** ![bjosve](https://avatars.discourse-cdn.com/v4/letter/b/ecae2f/32.png) [@bjosve](https://discuss.elastic.co/u/bjosve)\ **Post date:** [January 8, 2020, 3:06pm UTC](https://discuss.elastic.co/t/logstash-aggregate-filter-plugin-works-in-7-5-0-not-in-6-6-1/214260/3 "2020-01-08T15:06:45Z")
Thanks, I will try that and see if it stops working. UPDATE I did the opposite and added "pipeline.java\_execution: true" in 6.6.1 and now it works. It seems as it was implemented in 6.5 and set to false as default. Thank you very much! 🙂 /Bjorn ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [January 8, 2020, 4:00pm UTC](https://discuss.elastic.co/t/logstash-aggregate-filter-plugin-works-in-7-5-0-not-in-6-6-1/214260/4 "2020-01-08T16:00:40Z")
OK, but be aware that you are relying on a [bug](https://github.com/elastic/logstash/issues/10938) that will one day get fixed. ---
**Author:** ![bjosve](https://avatars.discourse-cdn.com/v4/letter/b/ecae2f/32.png) [@bjosve](https://discuss.elastic.co/u/bjosve)\ **Post date:** [January 9, 2020, 8:26am UTC](https://discuss.elastic.co/t/logstash-aggregate-filter-plugin-works-in-7-5-0-not-in-6-6-1/214260/5 "2020-01-09T08:26:31Z")
> [@Badger](#): > > OK, but be aware that you are relying on a [bug](https://github.com/elastic/logstash/issues/10938) that will one day get fixed. Yes, I will. But in this special environment we are going to use 6.6.1 for the foreseeable future. No updates at all. Thanks again. /Bjorn ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 6, 2020, 8:26am UTC](https://discuss.elastic.co/t/logstash-aggregate-filter-plugin-works-in-7-5-0-not-in-6-6-1/214260/6 "2020-02-06T08:26:32Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.