Incremental Grok Works, but logstash grok filter fails to parse simple literal string

Elastic Stack Logstash
1

Hello,

Trying to extract as many fields from my log messages, but running into some basic issues.

I have a log message like the following:

INFO 2016-07-21 13:17:48,139 [http-bio-8080-exec-5] com.vendor.recserver.controller.RestController - site:company; abtest:none; pagetemplate:PT_RelatedRec: Total Time = 1ms. widget:RecentlyViewedProduct time:0ms scanned:0 timebox:none fallback:0 of 5 widget:RelatedRec time:0ms scanned:4 timebox:none fallback:0 of 4 context-url:http://www.company.com/eu/p/347340

When I try to create a grok filter match to extract the site name "company" in the message , it works on incremental grok builder ( http://grokconstructor.appspot.com/do/constructionstep), but generates a grokparsefailure.

This config does not work:

filter {
date {
match => ["logtime", "ISO8601" ]
}
grok {
match => { "message" => "%{LOGLEVEL:severity} %{TIMESTAMP_ISO8601:logtime} %{NOTSPACE:javathread} %{JAVACLASS:class} - site:%{WORD:site} %{GREEDYDATA:therest}"
}
}
}

The next config works up to the java class, but as soon as I try to add the literal to skip over the literal string "- site:" I get the grokparsefailure tag. Also, is there a better way to use literal strings in my message to help match fields?

filter {
date {
match => ["logtime", "ISO8601" ]
}
grok {
match => { "message" => "%{LOGLEVEL:severity} %{TIMESTAMP_ISO8601:logtime} %{NOTSPACE:javathread} %{JAVACLASS:class} %{GREEDYDATA:therest}"
}
}
}

Thank you in advance for your help!

Cheers, Dario

2

match => { "message" => "%{LOGLEVEL:severity} %{TIMESTAMP_ISO8601:logtime} %{NOTSPACE:javathread} %{JAVACLASS:class} - site:%{WORD:site} %{GREEDYDATA:therest}"

This doesn't take into account that the actual log message has a semicolon after the site name. If you add that it'll work.

Also, is there a better way to use literal strings in my message to help match fields?

Not quite sure what you mean here.

3

Hello Magnus,

Thank you that worked. I think I now understand the difference when approaching a grok filter and using traditional regex.

Cheers, Dario