Grok: logstash package does not include default patterns

Elastic Stack Logstash
1

Hi,

I am currently running logstash-2.1.1 . While using grok patterns, I am not able to get the built-in logstash patterns for grok (like SPACE, NOTSPACE etc by default). As a result I had to download all the patterns into a directory and add the patterns_dir construct in every grok filter, which makes my config really long.

Is there a way to put all the grok patterns into any logstash directory so that the grok filter will pick them up by default instead of me having to define explicitly the patterns_dir in every grok construct ?

2

If you place your pattern files in the same directory as the standard pattern files things will work, but probably only until the logstash-patterns-core package is updated. If you specify patterns_dir Logstash should look both there and in the standard location. If that doesn't work, try enabling verbose log output with --verbose or even --debug to get additional clues about what's going on. Logstash will log all loaded patterns.

3

That's the question. what is the standard pattern files directory ?

I do not see the standard patterns directory in the logstash package I downloaded. I usually download the logstash tar.gz file from the downloads page

4

That's the question. what is the standard pattern files directory ?

Here:

$ tar ztf logstash-2.1.1.tar.gz | grep patterns/grok-patterns
logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-patterns-core-2.0.2/patterns/grok-patterns

But again, storing files there is most likely a bad idea.

1 Like
5

Thanks for that

Could you please expand on what you meant by but probably only until the logstash-patterns-core package is updated in your original reply

6

If you store the files in the directory of version 2.0.2 of the plugin your files will most likely not be used when you update to e.g. version 2.0.3.

7

Thanks . that helped