Parsing multiline logs in Logstash

Elastic Stack Logstash
1

I am using Logstash to parse my multiline logs. I have used multiline in filebeat. I am able to use grok filter for the first line of the log. But the pattern is not same for all multiline logs. For example, my logs are:

[timestamp] [bla] [bla] [[
A
at bla
at bla
B
C
at bla
]]

I want information given by A, B ,C. Position of A, B, C in the logs is not always same. Also, there are few logs in which A, B, C do not exist. Can I use grok pattern or anything to create fields from A, B, C? If yes, then how?

2

Maybe it's the aggregate plugin which you're looking for?
https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html

3

Can you configure FB or LS multiline to treat any line which does not begin with a timestamp to be multiline and fold it into the previous line.

That should get you the whole thing in one blob.

You could then test to see if A is in there and if so grok it. Repeat for B. Reapeat for C.

4

I know about multiline pattern. And Logstash will consider it as a single log. But the position of A, B, C is not fixed. Here is my problem. How will I know here is A and I will use conditions accordingly?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.