# How do i aggregate these documets
**URL:** https://discuss.elastic.co/t/how-do-i-aggregate-these-documets/210874
**Category:** Logstash
**Created:** [December 6, 2019, 11:16am UTC](https://discuss.elastic.co/t/how-do-i-aggregate-these-documets/210874 "2019-12-06T11:16:34Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@Aayush\_Agrawal](https://discuss.elastic.co/u/Aayush_Agrawal)
#### Post date: [December 6, 2019, 11:16am UTC](https://discuss.elastic.co/t/how-do-i-aggregate-these-documets/210874/1 "2019-12-06T11:16:34Z")
I have logs of around thousand lines in a file, from which only three lines are useful for me. I am using grok filter to identify and filter those logs, now i want them to be in same document. How do i do that?
I am running it for similar multiple files. I find path is only common field for all three documents of a single log file.
`
{
"message" =\> "[INFO main : 10/08/2019-23:49:43.286 : EDeliveryProcessor] COBDate: 10/08/2019",
"@version" =\> "1",
"SLA\_Start" =\> 2019-12-05T19:00:00.000Z,
"path" =\> "/opt/logstash/testlogs/runOfflinesC-DailyStmt.log",
"COBDate" =\> 2019-01-07T18:30:00.000Z,
"@timestamp" =\> 2019-12-06T10:35:58.510Z,
"host" =\> "[sd-0153-fc25.nam.nsroot.net](http://sd-0153-fc25.nam.nsroot.net)",
"sys" =\> "[INFO main : 10/08/2019-23:49:43.286 : EDeliveryProcessor]",
"SLA\_End" =\> 2019-12-05T19:30:00.000Z
}
{
"message" =\> " Offline batch Start Time -\>Oct 8, 2019 11:47:33 PM, End time -\>Oct 8, 2019 11:49:43 PM",
"@version" =\> "1",
"Start Time" =\> 2019-10-08T18:17:33.000Z,
"path" =\> "/opt/logstash/testlogs/runOfflinesC-DailyStmt.log",
"SLA\_Start" =\> 2019-12-05T19:00:00.000Z,
"End Time" =\> 2019-10-08T18:19:43.000Z,
"@timestamp" =\> 2019-12-06T10:35:58.509Z,
"host" =\> "[sd-0153-fc25.nam.nsroot.net](http://sd-0153-fc25.nam.nsroot.net)",
"SLA\_End" =\> 2019-12-05T19:30:00.000Z
}
{
"message" =\> "[INFO main : 10/08/2019-23:49:43.286 : EDeliveryProcessor] Production-C:DAR:DailyStmt: EDeliveryDriver2 (10/08/2019) : Processed \<176\> Users \<0\> Jobs Failed",
"@version" =\> "1",
"SLA\_Start" =\> 2019-12-05T19:00:00.000Z,
"path" =\> "/opt/logstash/testlogs/runOfflinesC-DailyStmt.log",
"date" =\> "10/08/2019",
"@timestamp" =\> 2019-12-06T10:35:58.510Z,
"host" =\> "[sd-0153-fc25.nam.nsroot.net](http://sd-0153-fc25.nam.nsroot.net)",
"sys" =\> "[INFO main : 10/08/2019-23:49:43.286 : EDeliveryProcessor]",
"Processed User" =\> 176,
"Failed User" =\> 0,
"SLA\_End" =\> 2019-12-05T19:30:00.000Z
}
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [December 6, 2019, 1:31pm UTC](https://discuss.elastic.co/t/how-do-i-aggregate-these-documets/210874/2 "2019-12-06T13:31:14Z")
You might be able to use an [aggregate](https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html#plugins-filters-aggregate-example3) filter based on [path].
---
### Author:  [@Aayush\_Agrawal](https://discuss.elastic.co/u/Aayush_Agrawal)
#### Post date: [December 9, 2019, 5:56am UTC](https://discuss.elastic.co/t/how-do-i-aggregate-these-documets/210874/3 "2019-12-09T05:56:02Z")
I need to give a code setting i am missing, can you help me writing aggregate filter part.
Currently I'm using below code -
aggregate{
task\_id =\> "%{path}"
}
As i have one field that is consistent and common ie Path, but there are many fields in document which are overlapping.
---
### Author:  [@Aayush\_Agrawal](https://discuss.elastic.co/u/Aayush_Agrawal)
#### Post date: [December 9, 2019, 11:47am UTC](https://discuss.elastic.co/t/how-do-i-aggregate-these-documets/210874/4 "2019-12-09T11:47:22Z")
What am i missing in this?
```
if "" in ['Start Time']{
aggregate{
task_id => "%{path}"
code => "
map['Start Time'] = event.get('Start Time');
map['End Time'] = event.get('End Time');
event.cancel()
"
map_action => create
}
}
if "" in ['Processed User']{
aggregate{
task_id => "%{path}"
code => "
map['Processed User'] = event.get('Processed User');
map['Failed User'] = event.get('Failed User');
event.cancel()
"
#map_action => update
}
}
if "" in ['COBDate']{
aggregate{
task_id => "%{path}"
code => "
map['COBDate'] = event.get('COBDate');
map['SLA_Start'] = event.get('SLA_Start');
map['SLA_End'] = event.get('SLA_End');
event.cancel()
#map_meta.timeout = 0
"
#map_action => update
push_previous_map_as_event => true
end_of_task => true
#timeout
}
}
```
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [December 9, 2019, 3:42pm UTC](https://discuss.elastic.co/t/how-do-i-aggregate-these-documets/210874/5 "2019-12-09T15:42:02Z")
> [@Aayush\_Agrawal](#):
>
> if "" in ['Start Time']{
What are you trying to test using that?
---
### Author:  [@Aayush\_Agrawal](https://discuss.elastic.co/u/Aayush_Agrawal)
#### Post date: [December 10, 2019, 9:38am UTC](https://discuss.elastic.co/t/how-do-i-aggregate-these-documets/210874/8 "2019-12-10T09:38:07Z")
I was using it wrong, if [Start Time] worked for me.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 7, 2020, 9:38am UTC](https://discuss.elastic.co/t/how-do-i-aggregate-these-documets/210874/9 "2020-01-07T09:38:14Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.