# Grok isn't loading patterns

**URL:** <https://discuss.elastic.co/t/grok-isnt-loading-patterns/100327>\
**Category:** Logstash\
**Created:** [September 13, 2017, 10:03am UTC](https://discuss.elastic.co/t/grok-isnt-loading-patterns/100327 "2017-09-13T10:03:38Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![nivo33](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nivo33/32/21866_2.png) [@nivo33](https://discuss.elastic.co/u/nivo33)\
**Post date:** [September 13, 2017, 10:03am UTC](https://discuss.elastic.co/t/grok-isnt-loading-patterns/100327/1 "2017-09-13T10:03:38Z")

</div>

Hi,

I have tried to implement some custom grok patterns into a separate file and load it into logstash conf by using patterns\_dir. Each time running logstash with the following conf output a pattern not defined error message. Using logstash 5.5

here is my pattern file "/etc/elk/logstash/patterns/temp\_patterns":

```
OPENSTACK_PROG (?:[a-zA-Z0-9_\-]+\.)+[A-Za-z0-9_\-$]+

REQ_LIST (\[(?:(req-%{UUID:request_id_list}|%{UUID:request_id_list}|%{BASE16NUM}|None|-|%{SPACE}))+\])?

```

(theres a newline between the patterns in the saved file, not sure if thats relevant)

and my conf:

```
input {
    beats {
        port => "5043"
        ssl => false
    }
}

filter{
  if "keystone" in [fields][tag] or [source] == "/var/log/keystone/keystone.log"{
    grok{
        patterns_dir => ["/etc/elk/logstash/patterns"]
        match => {"message" => "%{TIMESTAMP_ISO8601:timestamp} %{POSINT:openstack_pid} %{LOGLEVEL:level} %{OPENSTACK_PROG:openstack_program} %{REQ_LIST} %{WORD:verb} %{URI:URI}"}
        add_tag => ["found"]
    }
  }
}

output {
    elasticsearch {
        hosts => ["elk-elasticsearch:9201"]
    }

```

and the error that pops up:

```
[2017-09-13T09:48:49,027][ERROR][logstash.agent] Pipeline aborted due to error {:exception=># <Grok::PatternError: pattern %{OPENSTACK_PROG:openstack _program} not defined>, :backtrace=> ["/usr/share/logstash/vendor/bundle/jruby/1.9/gems/jls-grok-0.11.4/lib/grok-pure.rb:123:in `compile'", "org/jruby/RubyKerne l.java:1479:in `loop'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/jls-grok- 0.11.4/lib/grok-pure.rb:93:in `compile'", "/usr/share/logstash/vendor/bundle /jruby/1.9/gems/logstash-filter-grok- 3.3.1/lib/logstash/filters/grok.rb:274:in `register'", "org/jruby/RubyArray.java:1613:in `each'", "/usr/share/logstash/ven dor/bundle/jruby/1.9/gems/logstash-filter-grok-3.3.1/lib/logstash/filters/grok.rb:269:in `register'", "org/jruby/RubyHash.java:1342:in `each'", "/usr/share/log stash/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.3.1/lib/logstash/filters/grok.rb:264:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/ pipeline.rb:235:in `start_workers'", "org/jruby/RubyArray.java:1613:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:235:in `start_worke rs'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:188:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:302:in `start_pipelin e'"]} 
[2017-09-13T09:48:49,083][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
[2017-09-13T09:48:52,044][WARN][logstash.agent] stopping pipeline {:id=>"main"}

```

Any idea what is causing the issue?

Thanks

---

<div class="post-metadata">

**Author:** ![immavalls](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/immavalls/32/146501_2.png) [@immavalls](https://discuss.elastic.co/u/immavalls)\
**Post date:** [September 13, 2017, 3:51pm UTC](https://discuss.elastic.co/t/grok-isnt-loading-patterns/100327/2 "2017-09-13T15:51:04Z")

</div>

Newlines should not be relevant.  
As far as I can see it looks good.  
Does your logstash process run with a user that has file permissions to read the patterns file?

---

<div class="post-metadata">

**Author:** ![NerdSec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nerdsec/32/22056_2.png) [@NerdSec](https://discuss.elastic.co/u/NerdSec)\
**Post date:** [September 14, 2017, 1:14am UTC](https://discuss.elastic.co/t/grok-isnt-loading-patterns/100327/3 "2017-09-14T01:14:39Z")

</div>

Hi,

Patterns\_dir option should be a directory path, not a file path

Regards,  
N

---

<div class="post-metadata">

**Author:** ![NerdSec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nerdsec/32/22056_2.png) [@NerdSec](https://discuss.elastic.co/u/NerdSec)\
**Post date:** [September 14, 2017, 1:16am UTC](https://discuss.elastic.co/t/grok-isnt-loading-patterns/100327/4 "2017-09-14T01:16:58Z")

</div>

My bad, it is a directory in config.

Are you seeing the data in elasticsearch? Are the tags proper?

---

<div class="post-metadata">

**Author:** ![nivo33](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nivo33/32/21866_2.png) [@nivo33](https://discuss.elastic.co/u/nivo33)\
**Post date:** [September 14, 2017, 1:32pm UTC](https://discuss.elastic.co/t/grok-isnt-loading-patterns/100327/5 "2017-09-14T13:32:23Z")

</div>

Problem solved. I was running logstash through docker and the directories were configured differently there.  
Thanks!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 12, 2017, 1:32pm UTC](https://discuss.elastic.co/t/grok-isnt-loading-patterns/100327/6 "2017-10-12T13:32:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.