Multiline troubleshooting

[benpolzin]

I'm using Filebeat 5.0 and having some trouble with multiline for one particular logtype. The same multiline config is working well for some other logs on the same box. Here is my multiline config:

multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
multiline.negate: true
multiline.match: after
multiline.timeout: 60s  ##added this after reading another thread, didn't help

Here's a sample of the Filebeat log with debug enabled: https://gist.github.com/benpolzin/cc09a43ff82cc972bb4568b7fce53f2d

And here's an example of what the input log file looks like: https://gist.github.com/benpolzin/7aae6f2b971fe320df0a9ca02a86ed1e

I see this line in the debug:
Nov 4 08:04:22 hostName last message repeated 39 times

Is that an indication of multiline at work? It seems this might be related to my issue, but I'm not sure exactly what to do about it.

Thanks!

[andrewkroh]

I ran a quick test on the log file in your gist and it seems to as I would expect. Maybe there are some indentation problems with your config file.

./filebeat -c filebeat.dev.yml -e -d "*"

filebeat.prospectors:
- paths:
    - input.txt
  multiline:
    pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
    negate: true
    match: after

output.console.pretty: true

{
  "@timestamp": "2016-11-04T14:46:41.177Z",
  "beat": {
    "hostname": "macbook13.local",
    "name": "macbook13.local",
    "version": "5.0.0"
  },
  "input_type": "log",
  "message": "`2016-11-03T20:59:32.151-0500 [pool-51-thread-1] INFO  gov.nist.javax.sip.stack.SIPTransactionStack [] - \nfrom=10.27.2.81:5060\nto=10.27.2.30:5060\nisSender=true\ncallId=1705a4b9b8b8b6306c10ef741312894f@10.27.2.81\nmessage=\n\nOPTIONS sip:10.27.2.30 SIP/2.0\nCall-ID: 1705a4b9b8b8b6306c10ef741312894f@10.27.2.81\nCSeq: 0 OPTIONS\nFrom: \"Test User\" \u003csip:Test User@10.27.2.81\u003e;tag=356b873e\nTo: \u003csip:10.27.2.30\u003e\nVia: SIP/2.0/UDP 10.27.2.81:5060;branch=z9hG4bK-3138-45766235df5d77f306fb3bfc03d848f3\nMax-Forwards: 70\nContact: \"Test User\" \u003csip:Test User@10.27.2.81;transport=udp\u003e\nSupported: \nAccept: application/sdp\nUser-Agent: Test User 11.5.2\nContent-Length: 0\n\n",
  "offset": 650,
  "source": "input.txt",
  "type": "log"
}
{
  "@timestamp": "2016-11-04T14:46:41.178Z",
  "beat": {
    "hostname": "macbook13.local",
    "name": "macbook13.local",
    "version": "5.0.0"
  },
  "input_type": "log",
  "message": "2016-11-03T20:59:32.154-0500 [Thread-11515] INFO  gov.nist.javax.sip.stack.SIPTransactionStack [] - \nfrom=10.27.2.30:5060\nto=10.27.2.81:5060\nisSender=false\ncallId=1705a4b9b8b8b6306c10ef741312894f@10.27.2.81\nmessage=\n\nSIP/2.0 200 OK\nVia: SIP/2.0/UDP 10.27.2.81:5060;branch=z9hG4bK-3138-45766235df5d77f306fb3bfc03d848f3\nFrom: \"Test User\" \u003csip:Test User@10.27.2.81\u003e;tag=356b873e\nTo: \u003csip:10.27.2.30\u003e;tag=1383460837\nDate: Fri, 04 Nov 2016 01:59:32 GMT\nCall-ID: 1705a4b9b8b8b6306c10ef741312894f@10.27.2.81\nServer: Cisco-CUCM11.5\nCSeq: 0 OPTIONS\nAllow: INVITE,OPTIONS,INFO,BYE,CANCEL,ACK,PRACK,UPDATE,REFER,SUBSCRIBE,NOTIFY\nContent-Length: 0",
  "offset": 1286,
  "source": "input.txt",
  "type": "log"
}

[benpolzin]

Hi Andrew, Thanks for testing. Turns out I have too many lab boxes and this one was still stuck on Filebeat 1.3 with an updated filebeat.yml that used the new multiline.pattern format. Upgraded to 5.0 and all is well. Thanks!