Grok Pattern works in Grok debugger but fails in the parsing

The Grok Patter works in Grok debugger but fails to parse in logstash. I need help not sure whats wrong, I need to sort out ERROR logs separate Index and regular Debug into different index, My trying to get parsing done, but not sure, need some help.

When I test the patter in debugger it works
{
"package": "util.IConveUtility ",
"javaclass": "DAWSConnThread",
"log": " domainInitRqst::1::null::DDS_DATA_SOURCE_UNAVAILABLE::[[ERROR,Participant Error:null]]",
"action": " DAError",
"LEVEL": "ERROR",
"timestamp": "2019-07-03 03:06:10,043"
}

Input data

DEBUG 2019-07-03 02:58:42,024 [main] util.IConveUtility - Files are deleted..
DEBUG 2019-07-03 02:58:42,024 [main] util.IConveUtility - Output file created
ERROR 2019-07-03 03:06:10,043 [DAWSConnThread] App.DAErrorHandler - DAError: domainInitRqst::1::null::DDS_DATA_SOURCE_UNAVAILABLE::[[ERROR,Participant Error:null]]

input {
file{
path=> "c:/tmp/test.log"
start_position=>"beginning"
sincedb_path => "c:/tmp/null.sincedb"
}
}

filter {
grok{
match => { "message" => "%{LOGLEVEL:LEVEL} *%{TIMESTAMP_ISO8601:timestamp} [%{JAVACLASS:javaclass}] %{DATA:package}-%{DATA:action}\ *:%{GREEDYDATA:log}" }
}
}

output {
if "ERROR" in [LEVEL]
{file { path => "c:/tmp/test-error.txt" }}
file { path => "c:/tmp/test-error-2.txt" }
}

Your first problem is that your "JAVACLASS" does not match the JAVACLASS pattern, which requires two or more words separated using period.

I would use dissect to parse that.

dissect { mapping => { "message" => "%{LEVEL} %{timestamp} %{+timestamp} [%{javaclass}] %{package} - %{restOfLine}" } }

Then grok the [restOfLine] field.

If you want to go forward using grok make sure you understand why you should anchor your patterns to start of line if possible.

Thank you very much.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.