Multiline Parsing

[omer]

Hi,

I am trying to parse a multiline log file. Input comes from filebeat so i must use multiline settings at filebeat.yml. I searched for examples and made a configuration like that:

multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
multiline.negate: true
multiline.match: after

And here is my sample log file:

2017-09-21 11:32:05,361 INFO  [gov.nist.javax.sip.stack.SIPTransactionStack][dsipua-UDPMessageChannelThread-8] <message
from="10.6.64.110:5060" 
to="10.6.194.4:5060" 
time="1505971925361"
isSender="false" 
transactionId="z9hg4bk-373338-4e04f0bad7d0c0e443cda42f37f878fe" 
callId="d7ad85cc2c5f2ed0b28a6f11cb96b4e0@10.6.194.4" 
firstLine="SIP/2.0 100 Trying" 
>
<![CDATA[SIP/2.0 100 Trying
From: <sip:+000@10.6.194.4>;tag=3fe7bcf
To: <sip:+77784873684@ALMAS6.KCELL.KZ>
Via: SIP/2.0/UDP 10.6.194.4:5060;branch=z9hG4bK-373338-4e04f0bad7d0c0e443cda42f37f878fe;received=10.6.194.4
Call-ID: d7ad85cc2c5f2ed0b28a6f11cb96b4e0@10.6.194.4
CSeq: 1 INVITE
Content-Length: 0

]]>
</message>

However; filebeat sends them as seperate lines. Need help for this.

Thank you

[andrewkroh]

Double check that you have specified the multiline options in the proper location. I tested your multiline config against the sample log and it worked for me. The config should be intended like so:

filebeat.prospectors:
- paths:
    - 'input.log'
  multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
  multiline.negate: true
  multiline.match: after

(the important part is that multiline is at the same level as paths and the other prospector options)