I've been building some grok patterns to parse the log file and everything has been working fine. I create the grok patterns at http://grokdebug.herokuapp.com/ and sites show the patterns matching perfectly. I'm using logstash 2.1.1, elasticsearch 2.1.1 and kibana 4.3.0.
have then taken those patterns and implemented them with a filter on my logstash servers. Most of the filters work fine but GREEDYDATA is not matching for some reason.
Log file :
2016-10-07 16:46:42.4368 WindowsForm1.Form1 Error
Message : Input string was not in a correct format.
MethodName : Calculate
StackTrace : at System.Number.StringToNumber(String str, NumberStyles options, NumberBuffer& number, NumberFormatInfo info, Boolean parseDecimal)
at System.Number.ParseInt32(String s, NumberStyles style, NumberFormatInfo info)
at System.Convert.ToInt32(String value)
at WindowsForm1.Form1.Calculate(Object source, ElapsedEventArgs e) in d:\Roshan\WebLogg\WinLog\WindowsForm1\WindowsForm1\Form1.cs:line 59
grok pattern= %{TIMESTAMP_ISO8601:timestamp} %{NOTSPACE:ClassName} %{LOGLEVEL:logLevel}%{SPACE}%{GREEDYDATA:message}%{SPACE}%{GREEDYDATA:MethodName}%{SPACE}%{GREEDYDATA:StackTrace}
it give me output like :
{
"@timestamp" => "2016-10-07T11:16:42.436Z",
"message" => "Message : Input string was not in a correct format.\r\nMeth
odName : Calculate\r\nStackTrace : at System.Number.StringToNumber(String str
, NumberStyles options, NumberBuffer& number, NumberFormatInfo info, Boolean par
seDecimal)\r\n at System.Number.ParseInt32(String s, NumberStyles style, Numbe
rFormatInfo info)\r\n at System.Convert.ToInt32(String value)\r\n at Windows
Form1.Form1.Calculate(Object source, ElapsedEventArgs e) in d:\Roshan\WebLogg
\WinLog\WindowsForm1\WindowsForm1\Form1.cs:line 59\r\n\r",
"@version" => "1",
"tags" => [
[0] "multiline"
],
"host" => "DIN16002644",
"path" => "D:/ElasticStack/Data/file.log",
"type" => "VCC",
"timestamp" => "2016-10-07 16:46:42.4368",
"ClassName" => "WindowsForm1.Form1",
"logLevel" => "Error"
}
But i need Output like:
{
"@timestamp" => "2016-10-07T11:16:42.436Z",
"message" => "Message : Input string was not in a correct format.",
"@version" => "1",
"tags" => [
[0] "multiline"
],
"host" => "DIN16002644",
"path" => "D:/ElasticStack/Data/file.log",
"type" => "VCC",
"timestamp" => "2016-10-07 16:46:42.4368",
"ClassName" => "WindowsForm1.Form1",
"logLevel" => "Error",
"MethodName" =>"MethodName : Calculate",
"StackTrace" =>"StackTrace : at System.Number.StringToNumber(String str
, NumberStyles options, NumberBuffer& number, NumberFormatInfo info, Boolean par
seDecimal)\r\n at System.Number.ParseInt32(String s, NumberStyles style, Numbe
rFormatInfo info)\r\n at System.Convert.ToInt32(String value)\r\n at Windows
Form1.Form1.Calculate(Object source, ElapsedEventArgs e) in d:\Roshan\WebLogg
\WinLog\WindowsForm1\WindowsForm1\Form1.cs:line 59\r\n\r",
}
Please suggest me.