# Multiline codec

**URL:** https://discuss.elastic.co/t/multiline-codec/41260
**Category:** Logstash
**Created:** [February 9, 2016, 9:48am UTC](https://discuss.elastic.co/t/multiline-codec/41260 "2016-02-09T09:48:41Z")
**Posts on this page:** 16
**Page:** 1

<div class="post-metadata">

### Author: ![marcmga](https://avatars.discourse-cdn.com/v4/letter/m/3bc359/32.png) [@marcmga](https://discuss.elastic.co/u/marcmga)
#### Post date: [February 9, 2016, 9:48am UTC](https://discuss.elastic.co/t/multiline-codec/41260/1 "2016-02-09T09:48:41Z")

</div>

Hi,

I'm using multiline codec in my input configuration in order to merge lines in one "message".  
This lines are from a tomcat log. But as most of times it works, others doesn't. It mix logs from other "non-tomcat" logs from other servers (i guess).  
Why is mixing that? Is there a way to avoid this behaviour? I don't know if it's a best practise to use filter instead of input plugin.

Thanks

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 9, 2016, 10:41am UTC](https://discuss.elastic.co/t/multiline-codec/41260/2 "2016-02-09T10:41:35Z")

</div>

Please show your input configuration.

---

<div class="post-metadata">

### Author: ![marcmga](https://avatars.discourse-cdn.com/v4/letter/m/3bc359/32.png) [@marcmga](https://discuss.elastic.co/u/marcmga)
#### Post date: [February 9, 2016, 11:24am UTC](https://discuss.elastic.co/t/multiline-codec/41260/3 "2016-02-09T11:24:53Z")

</div>

Hi Magnus,

> input {  
> lumberjack {  
> port =\> 5043  
> ssl\_certificate =\> "/etc/pki/tls/certs/logstash-forwarder.crt"  
> ssl\_key =\> "/etc/pki/tls/private/logstash-forwarder.key"

> ```
> codec => multiline {
> pattern => "(^%{TIMESTAMP_ISO8601} ) | (^%{IPV4} )"
> negate => true
> what => "previous"
> }
> }
> 
> ```

> ```
> udp {
> port => 25826
> codec => collectd {}
> type => collectd
> }
> 
> ```
> 
> }

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 9, 2016, 12:10pm UTC](https://discuss.elastic.co/t/multiline-codec/41260/4 "2016-02-09T12:10:40Z")

</div>

Yeah, you can't really use the multiline codec for inputs that read events from multiple sources like lumberjack does. Switch to Filebeat and use it's recently introduced multiline feature.

---

<div class="post-metadata">

### Author: ![marcmga](https://avatars.discourse-cdn.com/v4/letter/m/3bc359/32.png) [@marcmga](https://discuss.elastic.co/u/marcmga)
#### Post date: [February 9, 2016, 12:35pm UTC](https://discuss.elastic.co/t/multiline-codec/41260/5 "2016-02-09T12:35:03Z")

</div>

Ahhh, ok Magnus.

What about using multiline filter? is a good option?

Thanks

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 9, 2016, 12:37pm UTC](https://discuss.elastic.co/t/multiline-codec/41260/6 "2016-02-09T12:37:10Z")

</div>

The multiline filter is deprecated. That said, with its [`stream_identity` option](https://www.elastic.co/guide/en/logstash/current/plugins-filters-multiline.html#plugins-filters-multiline-stream_identity) you should be able to use it.

---

<div class="post-metadata">

### Author: ![marcmga](https://avatars.discourse-cdn.com/v4/letter/m/3bc359/32.png) [@marcmga](https://discuss.elastic.co/u/marcmga)
#### Post date: [February 9, 2016, 3:12pm UTC](https://discuss.elastic.co/t/multiline-codec/41260/7 "2016-02-09T15:12:55Z")

</div>

Thanks Magnus,

However, i think that my two patterns are ot working.  
I mean: if i receive a IP address it stills adding the line to the previous message, which doesn't happen if i receive a timestamp.  
What's the way to user several patterns (A or B or C...)

Thank you!

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 9, 2016, 3:21pm UTC](https://discuss.elastic.co/t/multiline-codec/41260/8 "2016-02-09T15:21:03Z")

</div>

Start by changing

```
pattern => "(^%{TIMESTAMP_ISO8601} ) | (^%{IPV4} )"

```

to

```
pattern => "(^%{TIMESTAMP_ISO8601})|(^%{IPV4})"
```

---

<div class="post-metadata">

### Author: ![marcmga](https://avatars.discourse-cdn.com/v4/letter/m/3bc359/32.png) [@marcmga](https://discuss.elastic.co/u/marcmga)
#### Post date: [February 11, 2016, 2:03pm UTC](https://discuss.elastic.co/t/multiline-codec/41260/9 "2016-02-11T14:03:11Z")

</div>

Hi Magnus,

It works.  
But finally i moved to beats (as you recomended).  
What happens now is that this patterns (that i used in logstash multifilter) don't work for filebeat...  
Doesn't the patterns be the same?

Thanks again

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 11, 2016, 7:13pm UTC](https://discuss.elastic.co/t/multiline-codec/41260/10 "2016-02-11T19:13:21Z")

</div>

You're not showing your configuration so I'll have to guess: Filebeat's multiline support doesn't support the grok patterns that Logstash's multiline codec supports. You'll have to use standard regular expressions.

---

<div class="post-metadata">

### Author: ![marcmga](https://avatars.discourse-cdn.com/v4/letter/m/3bc359/32.png) [@marcmga](https://discuss.elastic.co/u/marcmga)
#### Post date: [February 12, 2016, 9:31am UTC](https://discuss.elastic.co/t/multiline-codec/41260/11 "2016-02-12T09:31:54Z")

</div>

Hi,

the patter i writed above.

> multiline:

> ```
> pattern: "(^%{TIMESTAMP_ISO8601})|(^%{IPV4})"
> negate: true
> match: after
> max_lines: 200
> timeout: 4s
> tail_files: true
> 
> ```

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 12, 2016, 9:37am UTC](https://discuss.elastic.co/t/multiline-codec/41260/12 "2016-02-12T09:37:15Z")

</div>

As I suspected you're trying to use grok patterns. Use a standard regexp instead.

---

<div class="post-metadata">

### Author: ![marcmga](https://avatars.discourse-cdn.com/v4/letter/m/3bc359/32.png) [@marcmga](https://discuss.elastic.co/u/marcmga)
#### Post date: [February 15, 2016, 4:14pm UTC](https://discuss.elastic.co/t/multiline-codec/41260/13 "2016-02-15T16:14:43Z")

</div>

Hello,

I'm trying with regexp but with no luck.  
Filebeats is not sending any data dute to multiline.  
I tried with this pattern  
^([0-9]{4})(-?)(1[0-2]|0[1-9])(-?)(3[01]|0[1-9]|[12][0-9])\s(2[0-3]|[01][0-9]):?([0-5][0-9]):?([0-5][0-9])$  
Just for the timestamp

---

<div class="post-metadata">

### Author: ![marcmga](https://avatars.discourse-cdn.com/v4/letter/m/3bc359/32.png) [@marcmga](https://discuss.elastic.co/u/marcmga)
#### Post date: [February 15, 2016, 4:39pm UTC](https://discuss.elastic.co/t/multiline-codec/41260/14 "2016-02-15T16:39:57Z")

</div>

I reply myself,

The problem was de \s which i changed for [[:space:]] however it multilines all mesages, even if the pattern is within the message...  
so i guess my pattern is not correct... no idea why.

The data i receiv is YYYY-MM-dd HH:mm:ss

Any idea?

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 15, 2016, 5:59pm UTC](https://discuss.elastic.co/t/multiline-codec/41260/15 "2016-02-15T17:59:56Z")

</div>

Please show your complete multiline configuration and an example message. Format everything as code.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 5:11am UTC](https://discuss.elastic.co/t/multiline-codec/41260/16 "2017-07-06T05:11:23Z")

</div>


