# Filebeat multiline codec not working in my case

**URL:** <https://discuss.elastic.co/t/filebeat-multiline-codec-not-working-in-my-case/85040>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [May 9, 2017, 8:44am UTC](https://discuss.elastic.co/t/filebeat-multiline-codec-not-working-in-my-case/85040 "2017-05-09T08:44:10Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![aarishramesh](https://avatars.discourse-cdn.com/v4/letter/a/dfb087/32.png) [@aarishramesh](https://discuss.elastic.co/u/aarishramesh)\
**Post date:** [May 9, 2017, 8:44am UTC](https://discuss.elastic.co/t/filebeat-multiline-codec-not-working-in-my-case/85040/1 "2017-05-09T08:44:10Z")

</div>

I have defined multiline codec in filebeat.yml like below

```
 multiline.pattern: '^%{TIMESTAMP_ISO8601} '
 multiline.negate: true
 multiline.match: after

```

But it does not seem to be working as multiple lines of log get appended together like below

Single line of log

```
2017-05-07 22:29:43 [0] [pool-2-thread-1] INFO c.app.task.ChannelActiveCheckTask - ---- 
Inside checkIfChannelActive execution ----

```

The corresponding log stored in elastic search after multi-line parsing

```
---- Inside checkIfChannelActive execution ---- 2017-05-09 08:16:13 [0] [pool-2-thread-1] INFO  
XYZZ - XYZ :: 

```

XYZ 2017-05-09 08:16:13 [0] [pool-2-thread-1] INFO XYZ - XYZYZZ

Since the above did not work, I also tried using the below multi-pattern but it does not work too  
multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}'

Below is my logstash.conf

input {  
beats {  
port =\> 5044  
}  
}

filter {  
mutate {  
gsub =\> ["message", "\n", " "]  
}  
grok {  
match =\> ["message", "%{TIMESTAMP\_ISO8601:timestamp} [%{NOTSPACE:uid}] [%  
{NOTSPACE:thread}] %{LOGLEVEL:loglevel} %{DATA:class}-%{GREEDYDATA:message}" ]  
overwrite =\> ["message"]  
}  
date {  
match =\> ["timestamp" , "yyyy-MM-dd HH:mm:ss"]  
target =\> "@timestamp"  
}  
if "\_grokparsefailure" in [tags] {  
drop { }  
}  
}

output {  
elasticsearch {  
hosts =\> localhost  
index =\> "%{type}-%{+YYYY.MM.dd}"  
}  
}

Can someonehelp me fix this ? This parsing works fine on applying multiline codec in logstash.conf

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [May 9, 2017, 2:10pm UTC](https://discuss.elastic.co/t/filebeat-multiline-codec-not-working-in-my-case/85040/2 "2017-05-09T14:10:20Z")

</div>

filebeat multiline pattern is not grok pattern as in logstash, but pure regular expressions. If you want to filter by date use:

```auto
multiline.pattern: '^\d{4}-\d{2}-\d{2} '

```

Note, grok like patterns are somewhat readable, but often having a more complex regex definition then just using a very simple regex. When doing multiline, you don't want to look at Content, but more at the shape of messages.

Do you have a more complete sample log for testing?

Also see this google playground for testing: [https://play.golang.org/p/ES47bG6XN4](https://play.golang.org/p/ES47bG6XN4)

By community you can also find the [filebeat-multiline-tester](https://github.com/hartfordfive/filebeat-multiline-tester).

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [May 9, 2017, 2:11pm UTC](https://discuss.elastic.co/t/filebeat-multiline-codec-not-working-in-my-case/85040/3 "2017-05-09T14:11:22Z")

</div>

Note: for testing I would start by disabling the filters in logstash and see filebeat is correctly creating the multiline events first (e.g. write events to file/console).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 6, 2017, 2:16pm UTC](https://discuss.elastic.co/t/filebeat-multiline-codec-not-working-in-my-case/85040/4 "2017-06-06T14:16:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.