What is the best workflow when developing Grok filters?
The docs advise to use http://grokdebug.herokuapp.com/.
Unfortunately the patterns on grokdebug seem outdated. They do not match logstash-patterns 4.0.2 that are packaged with Logstash 5.4.0:
$ docker run -it --rm logstash --version
$ docker run -it --rm --entrypoint logstash-plugin logstash list --verbose logstash-patterns-core
Are there some other tools - ideally official ones from elastic.co?
My current workflow is as follows:
- run logstash
- wait for logstash to start
- wait for logstash to process an example file
- look for "_grokparsefailure"
- edit grok filter match
- stop logstash
I recommend a tool I've written, Logstash Filter Verifier. It doesn't specifically target grok filters and your iterations won't be as fast as with the grokdebug site, but on the flip side you can test your whole filter setup against multiple inputs (so you don't tweak a filter to make Logstash parse one example file correctly and accidentally break another example file).
Unfortunately it hasn't yet been adapted for Logstash 5 yet (see https://github.com/magnusbaeck/logstash-filter-verifier/issues/8).
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.