Grok Debug (Workflow) with Logstash 5.4.0

[felixhummel]

Hi!

What is the best workflow when developing Grok filters?

The docs advise to use http://grokdebug.herokuapp.com/.

Unfortunately the patterns on grokdebug seem outdated. They do not match logstash-patterns 4.0.2 that are packaged with Logstash 5.4.0:

$ docker run -it --rm logstash --version
logstash 5.4.0
$ docker run -it --rm --entrypoint logstash-plugin logstash list --verbose logstash-patterns-core
logstash-patterns-core (4.0.2)

Are there some other tools - ideally official ones from elastic.co?

My current workflow is as follows:

• run logstash
• wait for logstash to start
• wait for logstash to process an example file
• look for "_grokparsefailure"
• edit grok filter match
• stop logstash
• repeat

Any advice?

Thanks!

Felix

[magnusbaeck (Magnus Bäck)]

I recommend a tool I've written, Logstash Filter Verifier. It doesn't specifically target grok filters and your iterations won't be as fast as with the grokdebug site, but on the flip side you can test your whole filter setup against multiple inputs (so you don't tweak a filter to make Logstash parse one example file correctly and accidentally break another example file).

Unfortunately it hasn't yet been adapted for Logstash 5 yet (see https://github.com/magnusbaeck/logstash-filter-verifier/issues/8).