I have one filebeat that reads severals different log formats. I have this settings.
One format that works just fine is a single liner, which is sent to Logstash as a single event.
Now, I have another format that is a multiliner. I want to read it as a single event and send it to Logstash for parsing.
This is the log format example, with two events.
error: I READ THIS. sent payload: [{"key": "values"}]
custom status response: [{"key1": "values"}]
callback headers: [{"key2": "values"}]
error stack: [ something really bad happened
at here loremisptul (/xx/xx/x)
at here loremisptul (/xx/xx/x)
at here loremisptul (/xx/xx/x)
at here loremisptul (/xx/xx/x)
at here loremisptul (/xx/xx/x)
at here loremisptul (/xx/xx/x)
at here loremisptul (/xx/xx/x)
at here loremisptul (/xx/xx/x)]
error: I AM NOT READING THIS. sent payload: [{"key": "values"}]
custom status response: [{"key1": "values"}]
callback headers: [{"key2": "values"}]
error stack: [ something really bad happened
at here loremisptul (/xx/xx/x)
at here loremisptul (/xx/xx/x)
at here loremisptul (/xx/xx/x)
at here loremisptul (/xx/xx/x)
at here loremisptul (/xx/xx/x)
at here loremisptul (/xx/xx/x)
at here loremisptul (/xx/xx/x)
at here loremisptul (/xx/xx/x)]
And here is the prospector configuration:
- input_type: log
paths: /Users/xxxxx/Downloads/elk/anotherlog/app.stderr.log
document_type: logsystemtwo
multiline.pattern: '`^=[A-Z]+|^$`'
multiline.negate: true
multiline.match: after
- input_type: log
paths: /Users/xxxxx/Downloads/elk/mylogs/access.log*
document_type: logsystemone
The issue is with the first multiline. It reads first event (I READ THIS), but ignores rest of it (I AM NOT READING THIS).
I have tried many different configurations, but I just cant get it to read other events.