# Logstash aggregate filter not working

**URL:** <https://discuss.elastic.co/t/logstash-aggregate-filter-not-working/68850>\
**Category:** Logstash\
**Created:** [December 13, 2016, 10:17am UTC](https://discuss.elastic.co/t/logstash-aggregate-filter-not-working/68850 "2016-12-13T10:17:27Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![azeemdin](https://avatars.discourse-cdn.com/v4/letter/a/91b2a8/32.png) [@azeemdin](https://discuss.elastic.co/u/azeemdin)\
**Post date:** [December 13, 2016, 10:17am UTC](https://discuss.elastic.co/t/logstash-aggregate-filter-not-working/68850/1 "2016-12-13T10:17:27Z")

</div>

Hi,  
I am using logstash logstash-2.4.1. I am running the instance using -w 1 flag as recommened for aggregate plugin. I am trying to parse my log file sample below

> **START** \*\*\*\*\*\*\*  
> Multi line  
> messsage  
> could be  
> XML  
> **END** \*\*\*\*\*\*\*\*

my aggregate configuration is as below

> filter {  
> if "START" in [message] {  
> aggregate {  
> task\_id =\> "%{file}"  
> code =\> "map['fullmessage'] = '';"  
> map\_action =\> "create"  
> }  
> }  
> else if "END" in [message] {  
> aggregate {  
> task\_id =\> "%{file}"  
> code =\> "event['message'] = map['fullmessage'];"  
> map\_action =\> "update"  
> add\_tag =\> ["aggregated"]  
> end\_of\_task =\> true  
> timeout =\> 90  
> }  
> }  
> else {  
> aggregate {  
> task\_id =\> "%{file}"  
> code =\> "map['fullmessage'] += event['message']"  
> map\_action =\> "update"  
> }  
> }  
> if "aggregated" not in [tags] {  
> drop {}  
> }  
> }

This is not generating any output, here is my output

> output {  
> stdout {codec=\>rubydebug }  
> }

Without aggregate it is working fine but problem is that I get each line as new event whereas I would like to take all lines as one event from START to END. Any help would be much appricated. Thanks.

---

<div class="post-metadata">

**Author:** ![azeemdin](https://avatars.discourse-cdn.com/v4/letter/a/91b2a8/32.png) [@azeemdin](https://discuss.elastic.co/u/azeemdin)\
**Post date:** [December 14, 2016, 9:59am UTC](https://discuss.elastic.co/t/logstash-aggregate-filter-not-working/68850/2 "2016-12-14T09:59:57Z")

</div>

For now I have used following multiline codec configuration in my input to merge all lines before END line.

```
codec => multiline {
  pattern => "(\\*END\\*)|(\\* END \\*)"
  what => "previous"
  negate => true
  auto_flush_interval => 10
}

```

Only problem is that the line with END text is generated as seperate event during flush i.e. I get two events as follows

> **START** \*\*\*\*\*\*\*  
> Multi line  
> messsage  
> could be  
> XML

And second is

> **END** \*\*\*\*\*\*\*\*

I can live with this for now 🙂 until someone provides a better solution.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 11, 2017, 10:00am UTC](https://discuss.elastic.co/t/logstash-aggregate-filter-not-working/68850/3 "2017-01-11T10:00:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.