Filebeat with multiline vs. multiline logstash codec

(Craig Foote) #1

I've seen some talk of a multiline option in filebeat but I'm in the process of setting up filebeat sending to logstash via beats input and need to parse multiline logs. Can I use the multiline codec in the input section for this?

(Magnus B├Ąck) #2

Yes, but only if Filebeat only monitors a single file. I strongly suggest that you use Filebeat's multiline feature (which will be available in Filebeat 1.1, slated for release next week but you can use a nightly build or build it yourself if you don't want to wait).

(Jerry Hoffmeister) #3

I stopped by to ask the same question. I have filebeat monitoring several files... is the multiline option a top level option or is / can it be per file?

(Jerry Hoffmeister) #4

nevermind - I've switched from the multiline filter in logstash to filebeat and it seems to be working just fine... kinda miss the grok support tho.

(Steffen Siering) #5

regarding grok the upcoming ingest-node feature might be of interest. E.g. see elasticsearch doc from master (development?) branch:

filebeat's task is to collect and ship logs. Any kind of additional event processing/routing should be done by Logstash or upcoming Ingest Node in elasticsearch.

(Jerry Hoffmeister) #6

One new issue I see is when using the logstash multiline filter the log line got "multiline" added to the tags array. After switching to filebeat multiline, I no longer see the tag?

(ruflin) #7

See this issue here:

(Vedran Maricevic) #8

It can be per file, in other words per prospector.

(system) #9