# Aggregating the log and delete the not required entries

**URL:** <https://discuss.elastic.co/t/aggregating-the-log-and-delete-the-not-required-entries/49536>\
**Category:** Logstash\
**Created:** [May 9, 2016, 10:17am UTC](https://discuss.elastic.co/t/aggregating-the-log-and-delete-the-not-required-entries/49536 "2016-05-09T10:17:02Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Gaurav\_Harsola](https://avatars.discourse-cdn.com/v4/letter/g/ecc23a/32.png) [@Gaurav\_Harsola](https://discuss.elastic.co/u/Gaurav_Harsola)\
**Post date:** [May 9, 2016, 10:17am UTC](https://discuss.elastic.co/t/aggregating-the-log-and-delete-the-not-required-entries/49536/1 "2016-05-09T10:17:02Z")

</div>

Hi,

Following are the content of my sample log file which has 2 fields cardNumber and Amount:  
825888372935 900  
825888372935 2900  
825888372936 800

After processing above log ,i want the following as my output  
825888372935 (900+2900)  
825888372936 800

Here is my approach

After applying some filter in my logstash conf file ,it generate the following output for above data.I have used aggregate function in my conf file something like :  
aggregate {  
task\_id =\> "%{CardNumber}"  
code =\> "map['sql\_duration'] ||= 0 ; map['sql\_duration'] += event['amount'];event['amount'] = map['sql\_duration']"  
}  
Output :

{  
"message" =\> "825888372935 900",  
"@version" =\> "1",  
"@timestamp" =\> "2016-05-09T10:07:53.017Z",  
"path" =\> "/home/logstash/test1.log",  
"host" =\> "Inspiron-3442",  
"CardNumber" =\> "825888372935",  
"amount" =\> 900  
}  
{  
"message" =\> "825888372935 2900",  
"@version" =\> "1",  
"@timestamp" =\> "2016-05-09T10:07:53.080Z",  
"path" =\> "/home/logstash/test1.log",  
"host" =\> "Inspiron-3442",  
"CardNumber" =\> "825888372935",  
"amount" =\> 3800  
}  
{  
"message" =\> "825888372936 800",  
"@version" =\> "1",  
"@timestamp" =\> "2016-05-09T10:07:53.081Z",  
"path" =\> "/home/logstash/test1.log",  
"host" =\> "Inspiron-3442",  
"CardNumber" =\> "825888372936",  
"amount" =\> 800  
}

Now i wan to push only accumulated amount incurred from a specific card number and delete the intermediate addition [record.In](http://record.In) above example i want to delete first result

{  
"message" =\> "825888372935 900",  
"@version" =\> "1",  
"@timestamp" =\> "2016-05-09T10:07:53.017Z",  
"path" =\> "/home/logstash/test1.log",  
"host" =\> "Inspiron-3442",  
"CardNumber" =\> "825888372935",  
"amount" =\> 900  
}

cause it was later added in next line where it matches the same [cardNumber.So](http://cardNumber.So),how can i remove the frist record before pushing into elasticsearch

Please help me out !!  
Thanks  
Gaurav

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:58am UTC](https://discuss.elastic.co/t/aggregating-the-log-and-delete-the-not-required-entries/49536/2 "2017-07-06T04:58:35Z")

</div>