Multiline - multiple level of multiline in Filebeat

Hi,

Can we use multiple level of multiline feature in Filebeat.

Input --> multiline (to join based on 1st pattern) --> processor(drop lines based on condition) -->multiline (to join based on 2pattern)

Input :
0 STARTED Thu Jan 30 22:48:00 2020
Starting process1
1 INFO Thu Jan 30 22:48:01 2020
process1 variable
2 STARTED Thu Jan 30 22:53:35 2020
process1 aborted.

Output (line to be send to Logstash)

  1. Multiline ouput
    |0|STARTED|Thu Jan 30 22:48:00 2020\n\tStarting process1|
    |---|---|---|
    |1|INFO|Thu Jan 30 22:48:01 2020\n\tprocess1 variable|
    |2|STARTED|Thu Jan 30 22:53:35 2020\n\tprocess1 aborted.|

  2. Processor output (Removing INFO)
    |0|STARTED|Thu Jan 30 22:48:00 2020\n\tStarting process1|
    |---|---|---|
    |2|STARTED|Thu Jan 30 22:53:35 2020\n\tprocess1 aborted.|

Till above I could achieve
Now next wanted above line in multiline and get output - a single line:

0 STARTED Thu Jan 30 22:48:00 2020\n\tStarting process1\n2 STARTED Thu Jan 30 22:53:35 2020\n\tprocess1 aborted.

Thanks & Regards,
Avijit Madal

Hi,

could you please format your post using </> ? It's really hard to read log entries.

----My Log--- (starting values 0,1,2..represent log lines spread into multiple lines)

0 STARTED Thu Jan 30 22:48:00 2020
Starting process1
1 INFO Thu Jan 30 22:48:01 2020
process1 variable
2 STARTED Thu Jan 30 22:53:35 2020
process1 aborted.

Expected output to logstash

0 STARTED Thu Jan 30 22:48:00 2020 Starting process1 2 STARTED Thu Jan 30 22:53:35 2020 process1 aborted.

Hi Marcin,

Let me know if second post makes it clear.
If not will try to rewrite the issue.

Thanks

Please rethink if you really need to build complex processing expressions. It makes it harder to maintain. It's easier sometimes to use a script processors: https://www.elastic.co/guide/en/beats/filebeat/master/processor-script.html

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.