Grok pattern creation error

Hello i am trying to make a grok pattern for that type of log:

[2020-05-13T13:31:42,545][DEBUG][logstash.filters.mutate ][filebeat-lasernet] filters/LogStash::Filters::Mutate: removing field {:field=>"message"}

but i receive grok parse failure.

can somebody may help?

Could you share more information about what you are trying to do? Also I've moved the topic to Logstash as I think you will get better support on this matter there.


Sorry for the late reply.

I am trying to parse that log:
[2020-05-13T13:31:42,545][DEBUG][logstash.filters.mutate ][filebeat-lasernet] filters/LogStash::Filters::Mutate: removing field {:field=>"message"}

In order to do that i use the following pattern

\[%{TIMESTAMP_ISO8601:logstash.log.timestamp}\]\[%{LOGSTASH_LOGLEVEL:log.level}\s?\]\[%{LOGSTASH_CLASS_MODULE:logstash.log.module}\s*\]\[%{WORD:logstash.log.pipeline_id}\] %{GREEDYMULTILINE:message}

But i receive an error:

How should i change my grok pattern?

I suspect you are using elastic ingest processor instead of logstash, can you clarify this?

You use custom patterns that seem related to ingest processors. I found these after a quick search:


The dot notation log.level is not standard for logstash, bracket notation should be instead [log][level].

Anyway, your last pattern doesn't work because WORD pattern doesn't match "filebeat-lasernet". You might use DATA pattern.

A tool like grok constructor is more useful to test patterns as it shows what has and has not matched.

Having said that, if your logs start with a fixed number of bracket fields, it would make more sense to use the dissect filter (for logstash) or dissect processor. It should perform much better and you can forget about patterns.

If you really need to use grok, at least you can start your match patterns with an anchor to the begining of the line, it will perform somewhat better. You can find more information here:

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.