# Aggregate filter dilemma

**URL:** <https://discuss.elastic.co/t/aggregate-filter-dilemma/259657>\
**Category:** Logstash\
**Created:** [December 26, 2020, 10:43am UTC](https://discuss.elastic.co/t/aggregate-filter-dilemma/259657 "2020-12-26T10:43:57Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![tommys](https://avatars.discourse-cdn.com/v4/letter/t/7993a0/32.png) [@tommys](https://discuss.elastic.co/u/tommys)\
**Post date:** [December 26, 2020, 10:43am UTC](https://discuss.elastic.co/t/aggregate-filter-dilemma/259657/1 "2020-12-26T10:43:58Z")

</div>

```
	Hi all,

```

hoping to get some feedback on this as it took me 3 days to resolve which I think is potentially a knowledge gap on my side or a BUG.

In summary:

1. Two logs, I want to aggregate using the aggregate filter. One field in common between both logs

```auto
    if "log1" in [tags] {

    		grok {
				match => ["message", "(?<log1_KEY>(?<=log1key:).*?(?=\s,))"]
			}
		
		
		aggregate {
		 task_id => "%{log1_KEY}"
		 .....
		 
		 }
		 
		 }
		 	 
		 
		if "log2" in [tags] {

    		grok {
				match => ["message", "(?<log2_KEY>(?<=log2key:).*?(?=\s,))"] # log2 key would be same as log1 key as they are correlated
			}
		
		
		aggregate {
		 task_id => "%{log2_KEY}"
		 .....
		 
		 }
		 
		 }

```

The issue here is that aggregate filter treats both log1\_key and log2\_key differently even though both contain the same value. They only way I got this work is by using the same name for key id such as log\_key.

So in summary I see the following when using aggregate fitler:

`log1_key = 123 log2_key = 123`

`log1_key == log2_key ---> aggregate filter fails to match. log_key == log_key --> works`

I am confused as to why this behaviour exists when the underlying values are the same?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [December 26, 2020, 3:27pm UTC](https://discuss.elastic.co/t/aggregate-filter-dilemma/259657/2 "2020-12-26T15:27:09Z")

</div>

> [@tommys](#):
>
> The issue here is that aggregate filter treats both log1\_key and log2\_key differently even though both contain the same value.

That is expected. The array of map elements is [indexed](https://github.com/logstash-plugins/logstash-filter-aggregate/blob/8603b168d93c43a4bbc72ca92a633d6c5276439e/lib/logstash/filters/aggregate.rb#L209) by both the value of task\_id and the sprintf'd value of task\_id. If the names of the two fields are different then they will be treated differently, even if the contents of the two fields are the same.

---

<div class="post-metadata">

**Author:** ![tommys](https://avatars.discourse-cdn.com/v4/letter/t/7993a0/32.png) [@tommys](https://discuss.elastic.co/u/tommys)\
**Post date:** [December 26, 2020, 3:43pm UTC](https://discuss.elastic.co/t/aggregate-filter-dilemma/259657/3 "2020-12-26T15:43:28Z")

</div>

thanks for the quick response. Is there any particular reason for this as this goes against all programming concepts (as far as I can think)?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [December 26, 2020, 3:54pm UTC](https://discuss.elastic.co/t/aggregate-filter-dilemma/259657/4 "2020-12-26T15:54:50Z")

</div>

> [@tommys](#):
>
> Is there any particular reason for this

I cannot speak to the programmer's thinking when they designed it, but to me it makes sense.

---

<div class="post-metadata">

**Author:** ![tommys](https://avatars.discourse-cdn.com/v4/letter/t/7993a0/32.png) [@tommys](https://discuss.elastic.co/u/tommys)\
**Post date:** [December 26, 2020, 10:58pm UTC](https://discuss.elastic.co/t/aggregate-filter-dilemma/259657/5 "2020-12-26T22:58:43Z")

</div>

Many thanks for the quick reply.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 23, 2021, 10:58pm UTC](https://discuss.elastic.co/t/aggregate-filter-dilemma/259657/6 "2021-01-23T22:58:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.