Filebeat single prospector with 2 different multiline pattern

Elastic Stack Beats
1

Hello All,

I would like to know if filebeat with single prospector can process 2 different logs with 2 different multiline patterns. If yes, How can i achieve that?

Problem :
The two below is a sample log format we have. The plan is to store these logs in the same index. These files are fetch in the same directory (so i suppose, we need to use just a single prospector right?). See below :

Log 1 log format
####Start###
Field1
Feild2
Feild3
####End###
####Start###
Field1
Feild2
Feild3
####End###

filebeat config

multiline.pattern: '##### START'
multiline.negate: true
multiline.match: after

Log 2 log format
Field1
Field2
Field3
, 7
Field1
Field2
Field3
, 7

filebeat config

multiline.pattern: ', 7'
multiline.negate: true
multiline.match: after

Now, we wanted to process this logs concurrently. How do we achieve that?

2

You can use and or expression (given negate and match are the same):

multiline.pattern: '^(##### START|, 7)'

All prospectors/harvester run concurrently. Each file is processed in isolation. What's wrong with having 2 prospector configs?

btw. multiline.pattern: '^##### Start' can be optimized by beats not to use the regular expression engine :slight_smile:

1 Like
3

Thank you so much! I tried it and it works! :smiley:

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.