"_grokparsefailure"

Elastic Stack Logstash
1

Hello,

I have a problem with one of my grok-filters. I know why this problem occur, but I don't know how I can fix it.

The grokparsefailure occur because of the smaller "<" and greater ">" characters within the message field.

The filter looks like: filter match => [ "message", "<%{BASE10NUM:}>%{MONTH:month} etc.

When I remove the < and > from the filter, the error do not occur anymore, but than the filter do not work any longer.

Can somebody give me a hint how I can fix the problem?

Greetz,
ABecker

2

Angle brackets are not metacharacters in regexps so this should be fine. Well, except that there's a spurious colon after BASE10NUM that you should remove (i.e. use %{BASE10NUM} instead of %{BASE10NUM:}). If that doesn't make a difference what do the input messages look like?

3

Hello Magnus,

thanks for you help, but your hint does not solve the problem.

This is my complete filter I am use:

filter {
grok {
match => [ "message", "<%{BASE10NUM}>%{MONTH:month}%{SPACE}%{MONTHDAY:day}%{SPACE}%{TIME:time}%{SPACE}%{HOSTNAME:server}%{SPACE}%{DATA:logfile}:%{SPACE}%{GREEDYDATA:message}" ]
overwrite => [ "message" ]
remove_field => [ "host" ]
}
csv {
columns => ["level","severity", "logger", "aspnet-sessionid", "organisation", "correlationState", "user", "msg", "exception", "method", "messageType"]
separator => ","
remove_field => [ "severity" ]
}
}

This is the output I receive from logstash:

{
"message" => [
[0] "Debug,10,SiebelThreadDispatcher,wwlv1ixzk35ypqo5oe20sjse,de,,user,Released slot for siebel webservice communication.,,SiebelThreadDispatcher,LogEntry"
],
"@version" => "1",
"@timestamp" => "2015-07-02T11:58:28.315Z",
"host" => "server",
"path" => "/tmp/test_oo.log",
"tags" => [
[0] "_grokparsefailure"
],
"level" => "Debug",
"logger" => "SiebelThreadDispatcher",
"aspnet-sessionid" => "wwlv1ixzk35ypqo5oe20sjse",
"organisation" => "de",
"correlationState" => nil,
"user" => "user",
"msg" => "Released slot for siebel webservice communication.",
"exception" => nil,
"method" => "SiebelThreadDispatcher",
"messageType" => "LogEntry"

I am pretty sure that this has to be correct. Also the online grok evaluator don't show me any error. But I still receive a grokparsefailure.

Greetz,

ABecker

4

What do the input strings look like? (I.e. what are you testing with the grok evaluator?)

5

Hello Magnus,

I found the problem within my configuration and fixed the problem. The problem was that, the events which are send from the origin logfile looking a little bit different to the logfiles I have got from the department to create the grok filter.

Now everything works fine.

Thank you very much.