I've been trying to get Filebeat (v1.1.1 on centos7) to apply a multiline filter before outputting a java log (utf-8 encoded) into redis. The java log looks like this:
2016-03-03 09:47:13,690 INFO Wall-e-main Robot - done processing, sleeping for 10 seconds
2016-03-03 09:47:19,101 INFO Wall-e-status ThreadedRobot - PoolSize: 18, ActiveCount: 0, QueueSize: 0
2016-03-03 09:47:23,696 INFO Wall-e-main ThreadedRobot - selected 0 articles for processing
2016-03-03 09:47:23,697 INFO Wall-e-main Robot - done processing, sleeping for 10 seconds
2016-03-03 09:47:29,039 INFO Roy-main Robot - refreshWorker articleIdKey null
2016-03-03 09:47:29,039 INFO Roy-main Robot - Return jedis connection to pool
There are also some java stack traces in there, which begin like this:
2016-03-03 09:46:52,562 ERROR Roy--Aggressive-worker-1 Dispatcher - RhinoException occurred while processing:
The filebeat config that i'm using is the following:
filebeat:
prospectors:
-
paths:
- /path/to/log
input_type: log
fields:
logtype: robot
environment: staging
multiline:
pattern: '^[[:digit:]]{4}-[[:digit:]]{2}-[[:digit:]]{2}'
negate: true
what: after
output:
redis:
host: "1.2.3.4"
port: 6379
With the multiline in place, I keep getting this error:
log.go:135: ERR Stop Harvesting. Unexpected encoding line reader error: unknown matcher type:
When I comment the multiline out, the file processes just fine. I already tried converting the file from utf-8 to ascii, which didn't do anything. The odd part is that the syslog, secure log and cron log process just fine with a multiline filter applied.