Question about multiline

Elastic Stack Logstash
1

Hi. I try to read multiline log from this script.

input {
file {
path => "/datatest/sample"
start_position => "beginning"
codec => multiline {
pattern => "^%{TIMESTAMP_ISO8601}"
what => "previous"
}
}
}
filter {
mutate {
gsub => [ "message", "r", "" ]
}
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:logdate} %{GREEDYDATA:message}" }
}
date {
match => [ "logdate" , ISO8601 ]
}
}
output {
stdout {}
}

My sample data is

2018-09-24-21.00.01.932752+420 I1A1576 LEVEL: Event
PID : 2294076 TID : 1 PROC : db2diag
INSTANCE: inst1 NODE : 000
HOSTNAME: svr1
EDUID : 1
FUNCTION: DB2 UDB, RAS/PD component, pdLogInternal, probe:120
START : New Diagnostic Log file
DATA #1 : Build Level, 232 bytes
Instance "rmsinpr1" uses "64" bits and DB2 code release "SQL10057"
with level identifier "0608010E".
Informational tokens are "DB2 v10.5.0.7", "s151221", "IP23949", Fix Pack "7".
DATA #2 : System Info, 416 bytes
System: AIX sermsdb02 1 7 00FA14E04C00
CPU: total:512 online:128 Cores per socket:10 Threading degree per core:8 SIMD:Y
Physical Memory(MB): total:102400 free:8870
Virtual Memory(MB): total:110592 free:16814
Swap Memory(MB): total:8192 free:7944
Kernel Params: msgMaxMessageSize:4194304 msgMaxQueueIDs:1048576
msgMaxQueueSize:4194304 shmMax:35184372088832 shmMin:1
shmIDs:1048576 shmSegments:268435456 semIDs:1048576
semNumPerID:65535 semOps:1024 semUndoPerProcess:1024
semMaxVal:32767 semAdjustOnExit:16384
Cur cpu time limit (seconds) = 0x7FFFFFFFFFFFFFFF
Cur file size limit (bytes) = 0x7FFFFFFFFFFFFFFF
Cur data size (bytes) = 0x7FFFFFFFFFFFFFFF
Cur stack size (bytes) = 0x0000000002000000
Cur core size (bytes) = 0x7FFFFFFFFFFFFFFF
Cur memory size (bytes) = 0x7FFFFFFFFFFFFFFF
nofiles (descriptors) = 0x7FFFFFFFFFFFFFFF

Information in this record is only valid at the time when this file was
created (see this record's time stamp)

2018-09-24-21.00.01.949833+420 I1578A395 LEVEL: Event
PID : 2294076 TID : 1 PROC : db2diag
INSTANCE: inst1 NODE : 000
HOSTNAME: svr1
EDUID : 1
FUNCTION: DB2 UDB, RAS/PD component, pdDiagArchiveDiagnosticLog, probe:88
CREATE : DB2DIAG.LOG ARCHIVE : /db2dump/db2diag.log_2018-09-24-21.00.01 : success
IMPACT : Potential

Out i got ->

{
"@version" => "1",
"path" => "/datatest/sample",
"message" => "PID : 2294076 TID : 1 PROC : db2diag",
"host" => "logstash",
"@timestamp" => 2019-06-29T10:11:58.593Z,
"tags" => [
[0] "_grokparsefailure"
]
}
{
"@version" => "1",
"path" => "/datatest/sample",
"message" => "Physical Memoy(MB): total:102400 fee:8870",
"host" => "logstash",
"@timestamp" => 2019-06-29T10:11:58.602Z,
"tags" => [
[0] "_grokparsefailure"
]
}
{
"@version" => "1",
"path" => "/datatest/sample",
"message" => "Swap Memoy(MB): total:8192 fee:7944",
"host" => "logstash",
"@timestamp" => 2019-06-29T10:11:58.603Z,
"tags" => [
[0] "_grokparsefailure"
]
}

The error were.

  1. First line was missing.
  2. grokparse fail.

What is my mistake ? Need help.

Thank you.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.