Filebeat - multiline pattern does not work as expected

Elastic Stack Beats
1

Hi,

I set up multiline.* properties for /etc/filebeat/filebeat.yml as bellow:
filebeat.inputs:
- type: log
paths:
- /home/eip/logs/*.log
tail_files: true
multiline.pattern: '^%{TIMESTAMP_ISO8601}'
multiline.negate: true
multiline.match: after

Here sample how above configuration group logs

multiline_problem
multiline_problem.JPG1384×479 132 KB

Unfortunatelly filebeat does not group lines as expected and is groupping much more than is should.
Is there maybe another configuration which cause the issue or is value of multiline.pattern property defined wrongly?

Thanks Mateusz

2

Could you post a sample log file and an example of your desired grouping here? For posting the sample log file you might need to use something like https://pastebin.com/ if it's too large. Once I have these I can do some experimentation to try and come up with the right multiline settings.

3

Hi @shaunak

Sure, let me please provide more details.
For logging we are using log4j with ConversattionPattern: %d [%t] %-5p - %m%n
So sample loog look like: https://pastebin.com/V1NnxMQc

whenever line in log file starts with pattern "%d [%t] %-5p" defined by log4j filebeat should group next lines till pattern occur again. Expected multiline logs will look like:

Thanks, Mateusz

4

Below pattern solved problem:
multiline.pattern: '^\d{4}-\d{2}-\d{2} '

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.