I am using Logstash to read some log file.
Here are some data sources records
<2016-07-07 00:31:01> Start
<2016-07-07 00:31:59> Warning - Export_Sysem 6 (1) => No records to be exported
<2016-07-07 00:32:22> Export2CICAP (04) => Export PO : 34 record(s)
<2016-07-07 00:32:22> Export2CICAP (04) => Export CO : 87 record(s)
<2016-07-07 00:32:22> Export2CICAP (04) => Export FC
This is my conf file
grok{
match => {"message" => [
'<%{TIMESTAMP_ISO8601:Timestamp}> (%{WORD:Level} - )%{NOTSPACE:Job_Code} => %{GREEDYDATA:Message}',
'<%{TIMESTAMP_ISO8601:Timestamp}> %{WORD:Parameter} - %{GREEDYDATA:Message}',
'<%{TIMESTAMP_ISO8601:Timestamp}> %{WORD:Status}',
]}
}
This is part of my output
{
"message" => "??2016-07-07 00:31:01> Start\r?",
"@version" => "1",
"@timestamp" => "2016-07-08T03:22:01.076Z",
"path" => "C:/CIGNA/Export.log",
"host" => "SIMSPad",
"type" => "txt",
"tags" => [
[0] "_grokparsefailure"
]
}
{
"message" => "<2016-07-07 00:31:59> Warning - Export_Sysem 6 (1) => No records to be exported\r?",
"@version" => "1",
"@timestamp" => "2016-07-06T16:31:59.000Z",
"path" => "C:/CIGNA/Export.log",
"host" => "SIMSPad",
"type" => "txt",
"Timestamp" => "2016-07-07 00:31:59",
"Parameter" => "Warning",
"Message" => "Export_Sysem 6 (1) => No records to be exported\r?"
}
{
"message" => "<2016-07-07 00:32:22> Export2CICAP (04) => Export CO : 87 record(s)\r?",
"@version" => "1",
"@timestamp" => "2016-07-06T16:32:22.000Z",
"path" => "C:/CIGNA/Export.log",
"host" => "SIMSPad",
"type" => "txt",
"Timestamp" => "2016-07-07 00:32:22",
"Status" => "Export2CICAP"
}
As seen from the output, the outcome did not fully parse the message. How should I modify my grok statement so it can fully parse the message?