Multiline codec query

dear all, Greetings!
I am trying to use logstash to parse and load the application logs and request expert advice her with multi line log msgs
using logstash 5.1.2

The log file starts with the below lines
totally 7 such lines that are form file header

Then the log message with 36 columns separated by | and beginning with | ( i am able to parse all these columns with the help of grok patterns)
There are lines in the below format between these log messages
xxxxx : TraceLog message somenumber

Below codec i tried.

codec => multiline
pattern => "^|"
negate => true
what => "previous"

It almost works, but

  1. All the 7 header msgs are getting appended into one, which is great. but sometimes the last log msg in the file is appended with this header msg.. not sure why, i m unable to find any trend why and when it happens! Any idea why it is happening and how can ths be prevented?
  2. the lines in btn the actual log messgs (like *: TraceLog message ) is getting appended to the previous log messg. Actually these msgs can be ignored. I am not finding a way how to identify these msgs alone and drop them! kindly suggest

really appreciate any help with this.. thanks much

I am thinking, if I can define the multiline pattern to parse the log file....
It is a new log msg if the line starts with "^|" or "FILE_TYPE:" or does not contain ":TraceLog message [NUMBER]"

Any ideas if this is possible, thanks

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.