Logstash input multiline codec Entire file is getting merged into single line

Hi All,

I am creating a logstash pipeline to process the following logs

My logstash config is

input{
file{
path => "E:/Grok_Practice/Practice/e.log"
start_position => "beginning"
sincedb_path => "LOG_NULL"
codec => multiline {
pattern => "^\d"
negate => true
what => "previous"
auto_flush_interval => 1
}
}
}

The output is the entire log lines are getting merged into a single line as follows

"tags" => [
[0] "multiline"
],
"message" => "12546\tMon Mar 30 08:02:36.902032\tipcmisc.c355\r\tIPC1300002 - INFO: JDEIPC Initialized semaphore array: key=8002, numSems=1000, id=27\r\r12546\tMon Mar 30 08:02:36.932640\tipcmisc.c355\r\tprocess 12546 <E910SYS/JDENET_K> registered in entry 60\r\r12546\tMon Mar 30 08:02:36.974264\tnetcfg.c297\r\tStarting Kernel of Type:CALL OBJECT KERNEL\r\r12546\tMon Mar 30 08:02:37.905752\txmlrequest.cpp1335\r\tICU0000017 - ICU CodePage for 37 is ibm-37.\r\r12546\tMon Mar 30 08:02:37.911968\tjdekdisp.c2735\r\tINITIALIZING CALL OBJECT KERNEL\r\r12546\tMon Mar 30 08:02:37.917520\twinansi.c1420\r\tLIB0000572 - WARNING!!! [SECURITY][Password] Password not encrypted in the INI\r\r12546\tMon Mar 30 08:02:37.932992\twinansi.c1420\r\tLIB0000572 - WARNING!!! [SECURITY][Password] Password not encrypted in the INI\r\r12546\tMon Mar 30 08:02:38.276624\twinansi.c1420\r\tLIB0000572 - WARNING!!! [SECURITY][Password] Password not encrypted in the INI\r\r12546\tMon Mar 30 08:02:38.278176\tjdekdisp.c2824\r\tKNT0000888 - Call Object Kernel Thread Pool in multi-threaded mode.\r\r12546\tMon Mar 30 08:02:38.278240\tjdekdisp.c2831\r\tKNT0000999 - Call Object Kernel Thread Pool Setting: size 20, increment 5\r\r"

I am stuck on this. This works for me if I copy the contents of the inputs file at that time end of line character is LF. But directly copying the file from the AS 400 server it is not working because the end of character is CR.

Is there anyway to handle this?

Any suggestions for the above would be more helpful for me.

Thanks in advance,
Ranjith

You could use the delimiter option on the file input to tell it what line endings are in the file.

If you are running on Windows and do not want the file input to persist the in-memory sincedb across restarts then set 'sincedb_path => "NUL"'.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.