Dissect first then multiline in filebeat

Elastic Stack Beats
1

Hi everyone,

Any help would be greatly appreciated.

We recently upgraded our kubernetes cluster and all logs are now coming in with the new kubernetes CRI log format, that includes a timestamp and the output stream type prepending every log event example:

2023-10-06T00:17:09.669794202Z stdout F Exception in thread "main" java.lang.NullPointerException
2023-10-06T00:17:09.669794202Z stdout F         at com.example.myproject.Book.getTitle(Book.java:16)
2023-10-06T00:17:09.669794202Z stdout F         at com.example.myproject.Author.getBookTitles(Author.java:25)
2023-10-06T00:17:10.113242941Z stdout F         at com.example.myproject.Bootstrap.main(Bootstrap.java:14)

I am able to sucessfully remove the new CRI logs by using the dissect filter in Filebeat on my filestream input, to extract the log messages before sending it to Logstash.

However, I'm running into issues with multiline log events. From my testing, I haven't been able to first extract the log messages using dissect, then apply the multiline parser after to aggregate log events. Multiline never seems to go through as I am assuming it is being applied to log events before it's been dissected.

Does anyone know if running multiline after dissect is possible? Thanks in advance for your time and any help

2

Any help would be greatly appreciated :pray:

3

I don't think so, the parsers are executed before the processors.

But, wouldn't using this configuration works?

parsers:
  - container:
     # Source of container events. Available options: all, stdin, stderr.
     stream: all
     # Format of the container events. Available options: auto, cri, docker, json-file
     format: cri

I do not use k8s or filebeat, so I can not test it, but it's worth a try.