# Multiline issue

**URL:** https://discuss.elastic.co/t/multiline-issue/33349
**Category:** Logstash
**Created:** [October 30, 2015, 7:38am UTC](https://discuss.elastic.co/t/multiline-issue/33349 "2015-10-30T07:38:00Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![11191](https://avatars.discourse-cdn.com/v4/letter/1/9dc877/32.png) [@11191](https://discuss.elastic.co/u/11191)
#### Post date: [October 30, 2015, 7:38am UTC](https://discuss.elastic.co/t/multiline-issue/33349/1 "2015-10-30T07:38:00Z")

</div>

Hi everyone. In Logstash 1.5 I had this config working:

input {  
lumberjack {  
port =\> 5000  
ssl\_certificate =\> "/certs/logstash-forwarder.crt"  
ssl\_key =\> "/certs/logstash-forwarder.key"  
}  
}

filter {  
if [type] == "type1" or [type] == "type2" {  
multiline {  
pattern =\> "^\d{4}:\d{2}:\d{2}"  
negate =\> true  
what =\> "previous"  
}  
} else if [type] == "type3" or [type] == "type4" {  
multiline {  
pattern =\> "^####"  
negate =\> true  
what =\> "previous"  
}  
}  
}

output {  
elasticsearch { hosts =\> "localhost:8200" }  
stdout { codec =\> rubydebug }

but now in Logstash 2.0 it complains about multiline in filter section, so I guess I have to switch to codec multiline in input section, but there are no IFs.

How can I manage multilining of different types of logs?

P/S/ types are set in logstash-forwarder on multiple hosts

---

<div class="post-metadata">

### Author: ![suyograo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/suyograo/32/44898_2.png) [@suyograo](https://discuss.elastic.co/u/suyograo)
#### Post date: [November 3, 2015, 12:58am UTC](https://discuss.elastic.co/t/multiline-issue/33349/2 "2015-11-03T00:58:30Z")

</div>

> [@11191](#):
>
> input {lumberjack {port =\> 5000ssl\_certificate =\> "/certs/logstash-forwarder.crt"ssl\_key =\> "/certs/logstash-forwarder.key"}}
> 
> filter {if [type] == "type1" or [type] == "type2" {multiline {pattern =\> "^\d{4}:\d{2}:\d{2}"negate =\> truewhat =\> "previous"}} else if [type] == "type3" or [type] == "type4" {multiline {pattern =\> "^####"negate =\> truewhat =\> "previous"}}}
> 
> output {elasticsearch { hosts =\> "localhost:8200" }stdout { codec =\> rubydebug }

@11191 please use `-w 1` flag when you start Logstash to force the # of workers to 1. By default in 2.0, we set the number of workers to half of the # of cores, but since you are using multiline filter which is not thread safe, you'll have to go back to 1 filter workers.

I'll update our documentation

---

<div class="post-metadata">

### Author: ![erikstephens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/erikstephens/32/5430_2.png) [@erikstephens](https://discuss.elastic.co/u/erikstephens)
#### Post date: [November 13, 2015, 7:56pm UTC](https://discuss.elastic.co/t/multiline-issue/33349/3 "2015-11-13T19:56:49Z")

</div>

I think I'm seeing similar issue with the aggregate filter. It's using a mutex but the threads still seem to be stepping on each other. I'm working on an unrelated change to that plugin and could maybe address this as well but I'm not much of a rubyist. Is there a recommended way to handle multi-line/multi-event processing in filter plugins?

---

<div class="post-metadata">

### Author: ![fbaligand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fbaligand/32/5657_2.png) [@fbaligand](https://discuss.elastic.co/u/fbaligand)
#### Post date: [November 14, 2015, 11:48am UTC](https://discuss.elastic.co/t/multiline-issue/33349/4 "2015-11-14T11:48:46Z")

</div>

I confirm that "-w 1" option is recommended for aggregate filter.

Even if aggregate code is technically protected against multithreaded access, multiple events at the same time mean you don't have control on events order.  
And as aggregate filter is dependent on events order to do its job, it requires only one worker to be sure that it behaves nicely.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 5:22am UTC](https://discuss.elastic.co/t/multiline-issue/33349/5 "2017-07-06T05:22:49Z")

</div>


