# Aggregate Performing Unpredictably

**URL:** <https://discuss.elastic.co/t/aggregate-performing-unpredictably/251336>\
**Category:** Logstash\
**Created:** [October 7, 2020, 7:38pm UTC](https://discuss.elastic.co/t/aggregate-performing-unpredictably/251336 "2020-10-07T19:38:27Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![NomadicCodeGuy](https://avatars.discourse-cdn.com/v4/letter/n/7cd45c/32.png) [@NomadicCodeGuy](https://discuss.elastic.co/u/NomadicCodeGuy)\
**Post date:** [October 7, 2020, 7:38pm UTC](https://discuss.elastic.co/t/aggregate-performing-unpredictably/251336/1 "2020-10-07T19:38:27Z")

</div>

I am using the aggregate filter plugin to copy a field to all entries from the same file path that have a "nil" entry for that field. It works perfectly for around 95 percent of the time. The strange thing is that with exactly the same .conf file settings and using the same training data, a different number of fields are successfully copied each time. I'm sure there is some underlying issue with my implementation but I am not seeing it myself. Here is the settings I am using for my aggregate filter plugin. Note that the entries that have the scenario data always come before the nil scenarios.

```auto
if [scenario] != "" #scenario field detected, add to map for that file path
{
	aggregate 
	{
		 task_id => "%{[log][file][path]}"
		 code => "map['scenario'] = event.get('scenario')"
	}
}
if [scenario] == "" #nil scenario detected, add mapped data to it's scenario field
{
	aggregate 
	{
		 task_id => "%{[log][file][path]}"
		 code => "event.set('scenario', map['scenario'])"
	}
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [October 7, 2020, 7:57pm UTC](https://discuss.elastic.co/t/aggregate-performing-unpredictably/251336/2 "2020-10-07T19:57:57Z")

</div>

You have set pipeline.workers to 1, right?

---

<div class="post-metadata">

**Author:** ![NomadicCodeGuy](https://avatars.discourse-cdn.com/v4/letter/n/7cd45c/32.png) [@NomadicCodeGuy](https://discuss.elastic.co/u/NomadicCodeGuy)\
**Post date:** [October 7, 2020, 8:05pm UTC](https://discuss.elastic.co/t/aggregate-performing-unpredictably/251336/3 "2020-10-07T20:05:21Z")

</div>

Yes. I was able to fix the issue by switching to this aggregate filter configuration using if-else instead of two independent if's.

```auto
if [scenario] #scenario field detected, add to map for that file path
	{
		aggregate 
		{
		 task_id => "%{[log][file][path]}"
		 code => "map['scenario'] = event.get('scenario')"
		}
	}
else #add map data to it's scenario field
	{
		aggregate 
		{
		 task_id => "%{[log][file][path]}"
		 code => "event.set('scenario', map['scenario'])"
		}
	}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 4, 2020, 8:05pm UTC](https://discuss.elastic.co/t/aggregate-performing-unpredictably/251336/4 "2020-11-04T20:05:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.