# Logstash aggregate without unique id
**URL:** https://discuss.elastic.co/t/logstash-aggregate-without-unique-id/257068
**Category:** Logstash
**Created:** [November 30, 2020, 1:41pm UTC](https://discuss.elastic.co/t/logstash-aggregate-without-unique-id/257068 "2020-11-30T13:41:28Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@bbwolf](https://discuss.elastic.co/u/bbwolf)
#### Post date: [November 30, 2020, 1:41pm UTC](https://discuss.elastic.co/t/logstash-aggregate-without-unique-id/257068/1 "2020-11-30T13:41:29Z")
Hello, supposed il have current log
```auto
INFO - 12345 - TASK_START - start
INFO - 12345 - SQL - sqlQuery1 - 12
INFO - 12345 - SQL - sqlQuery2 - 34
INFO - 12345 - TASK_END - end
INFO - 12345 - TASK_START - start
INFO - 12345 - SQL - sqlQuery1 - 24
INFO - 12345 - SQL - sqlQuery2 - 48
INFO - 12345 - TASK_END - end
```
What Il would do is to use logstash aggregate plugin for this where Il don't have unique id.
For this lines of log, my desired output is two documents 🙂
```auto
INFO - 12345 - 46 (Result of 12+ 34)
INFO - 12345 - 72 ( Result offre 24+48)
```
Thanks in advance.
Regards
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 30, 2020, 3:28pm UTC](https://discuss.elastic.co/t/logstash-aggregate-without-unique-id/257068/2 "2020-11-30T15:28:02Z")
That exactly matches the use case in [example 1](https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html#plugins-filters-aggregate-example1) of the documentation.
---
### Author:  [@bbwolf](https://discuss.elastic.co/u/bbwolf)
#### Post date: [November 30, 2020, 3:47pm UTC](https://discuss.elastic.co/t/logstash-aggregate-without-unique-id/257068/4 "2020-11-30T15:47:29Z")
When I use example 1 configuration, This give me one document
INFO - 12345 - 118 ( result of 12+34+24+48)
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 30, 2020, 3:50pm UTC](https://discuss.elastic.co/t/logstash-aggregate-without-unique-id/257068/5 "2020-11-30T15:50:47Z")
That surprises me, since the [code](https://github.com/logstash-plugins/logstash-filter-aggregate/blob/8603b168d93c43a4bbc72ca92a633d6c5276439e/lib/logstash/filters/aggregate.rb#L236) deletes the map entry if end\_of\_task is true.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 30, 2020, 4:15pm UTC](https://discuss.elastic.co/t/logstash-aggregate-without-unique-id/257068/6 "2020-11-30T16:15:29Z")
You can get it to work using `--pipeline.batch.size 1`.
By default, logstash works in batches of 125 events, so 125 events are parsed using grok, then 125 events go through the aggregate that creates the map if it does not exist, then 125 events update the map, then 125 events go through the aggregate that ends the aggregation if it is a TASK\_END. So the sql\_duration with the value of 118 is added to the _first_ TASK\_END, not the second.
---
### Author:  [@bbwolf](https://discuss.elastic.co/u/bbwolf)
#### Post date: [November 30, 2020, 5:06pm UTC](https://discuss.elastic.co/t/logstash-aggregate-without-unique-id/257068/7 "2020-11-30T17:06:17Z")
Great, it works Fine.
Thank you
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 28, 2020, 5:06pm UTC](https://discuss.elastic.co/t/logstash-aggregate-without-unique-id/257068/8 "2020-12-28T17:06:30Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.