dear all, Greetings!
I am trying to use logstash to parse and load the application logs and request expert advice her with multi line log msgs
using logstash 5.1.2
The log file starts with the below lines
FILE_TYPE:
Encoding:
Record_seperator:
totally 7 such lines that are form file header
Then the log message with 36 columns separated by | and beginning with | ( i am able to parse all these columns with the help of grok patterns)
There are lines in the below format between these log messages
xxxxx : TraceLog message somenumber
Below codec i tried.
codec => multiline
{
pattern => "^|"
negate => true
what => "previous"
}
It almost works, but
- All the 7 header msgs are getting appended into one, which is great. but sometimes the last log msg in the file is appended with this header msg.. not sure why, i m unable to find any trend why and when it happens! Any idea why it is happening and how can ths be prevented?
- the lines in btn the actual log messgs (like *: TraceLog message ) is getting appended to the previous log messg. Actually these msgs can be ignored. I am not finding a way how to identify these msgs alone and drop them! kindly suggest
really appreciate any help with this.. thanks much