# Filebeat vs logstash for handling multiline
**URL:** https://discuss.elastic.co/t/filebeat-vs-logstash-for-handling-multiline/81713
**Category:** Logstash
**Created:** [April 9, 2017, 2:09pm UTC](https://discuss.elastic.co/t/filebeat-vs-logstash-for-handling-multiline/81713 "2017-04-09T14:09:32Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@santhosh244](https://discuss.elastic.co/u/santhosh244)
#### Post date: [April 9, 2017, 2:09pm UTC](https://discuss.elastic.co/t/filebeat-vs-logstash-for-handling-multiline/81713/1 "2017-04-09T14:09:32Z")
Hi,
We have 40 instances sending logs parallely to centralized logstash via filebeat. We have tried multiline codec with logstash and filebeat both as well. Both seems to be working.
But we see few issues with logstash like filebeat queues files when log rotation happens during high volume duration. (we run logstash with default worker i.e 1).
Is there any recommendation for handling multiline messages should we use filebeat or logstash?
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 9, 2017, 2:23pm UTC](https://discuss.elastic.co/t/filebeat-vs-logstash-for-handling-multiline/81713/2 "2017-04-09T14:23:02Z")
Performing multiline processing as close to the source as possible is generally preferable, so I would recommend doing it in Filebeat. This will allow you to send data either to a message queue or load balance across multiple Logstash instances without having to worry about lines that are supposed to be merged at a later stage get split up.
---
### Author:  [@TheoAndersen](https://discuss.elastic.co/u/TheoAndersen)
#### Post date: [April 9, 2017, 3:10pm UTC](https://discuss.elastic.co/t/filebeat-vs-logstash-for-handling-multiline/81713/3 "2017-04-09T15:10:55Z")
But the multiline possibilities in filebeat is only the one that merges specific line endings right? The one meant for exceptions and the like?
Is there a way to set this up for matching a start-line + end-line (sorry if this hijacks this post)
---
### Author:  [@santhosh244](https://discuss.elastic.co/u/santhosh244)
#### Post date: [April 9, 2017, 3:14pm UTC](https://discuss.elastic.co/t/filebeat-vs-logstash-for-handling-multiline/81713/4 "2017-04-09T15:14:24Z")
Thanks for your swift response ๐ Because i don't see any discussion on this topic. So just wanted to know more details on this comparison.
---
### Author:  [@santhosh244](https://discuss.elastic.co/u/santhosh244)
#### Post date: [April 9, 2017, 3:26pm UTC](https://discuss.elastic.co/t/filebeat-vs-logstash-for-handling-multiline/81713/5 "2017-04-09T15:26:16Z")
multiline for filebeat works for start line as well. For application traceback it works. Just a slight difference in syntax from logstash grok. Will post you some examples soon for python traceback.
---
### Author:  [@TheoAndersen](https://discuss.elastic.co/u/TheoAndersen)
#### Post date: [April 9, 2017, 3:34pm UTC](https://discuss.elastic.co/t/filebeat-vs-logstash-for-handling-multiline/81713/6 "2017-04-09T15:34:28Z")
That would be great. Thanks
It's just something like
Event\_starts
EventContent
EventContent
Event\_ends
Then matching the start and end words (or just the start) to merge into a single event from which fields can be extracted in the content lines
Just haven't 'gotten' the configs yet I guess
---
### Author:  [@TheoAndersen](https://discuss.elastic.co/u/TheoAndersen)
#### Post date: [April 9, 2017, 5:18pm UTC](https://discuss.elastic.co/t/filebeat-vs-logstash-for-handling-multiline/81713/7 "2017-04-09T17:18:56Z")
Inspired by your suggestion on filbert/multiline, I've dug a bit deeper, and found that it can, as you said, merge multiline based on a start-line, if you use it something like this
> ```
> multiline
> - pattern: "start-line-pattern"
> - negate: true
> - match: after
>
> ```
Only slight thing i haven't worked out, is how to make it match the end of an event.
This is because this multiline configuration will continue waiting for the next "start-line-pattern", at which it will then emit the current event, and start another.. But then the last event will not be emitted, until the given timeout is reached.
Is there anyway to specify a flush pattern, ending the multiline?
something like:
> ```
> multiline
> - pattern: "start-line-pattern"
> - negate: true
> - match: after
> - flush-pattern: "end-line-pattern"
>
> ```
That would be pretty cool ๐
edit: I filed a suggestion for this on the beats GitHub page ([Filebeat suggestion - allowing for something like multiline.flush\_pattern? ยท Issue #3964 ยท elastic/beats ยท GitHub](https://github.com/elastic/beats/issues/3964))
Please let me know if this is possible already ๐
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 7, 2017, 5:20pm UTC](https://discuss.elastic.co/t/filebeat-vs-logstash-for-handling-multiline/81713/8 "2017-05-07T17:20:45Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.