# Custom Pattern in grok filter

**URL:** <https://discuss.elastic.co/t/custom-pattern-in-grok-filter/129413>\
**Category:** Logstash\
**Created:** [April 25, 2018, 6:14am UTC](https://discuss.elastic.co/t/custom-pattern-in-grok-filter/129413 "2018-04-25T06:14:48Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Akash\_Tanwar](https://avatars.discourse-cdn.com/v4/letter/a/3ab097/32.png) [@Akash\_Tanwar](https://discuss.elastic.co/u/Akash_Tanwar)\
**Post date:** [April 25, 2018, 6:14am UTC](https://discuss.elastic.co/t/custom-pattern-in-grok-filter/129413/1 "2018-04-25T06:14:49Z")

</div>

Hi, I am very new to ELK. I was trying to apply inline custom pattern to my log file but my approach is giving me errors. Can someone help me on this?

template of logs:  
172.31.29.134 - - [02/Feb/2018:06:25:05 +0000] "GET /favicon.ico HTTP/1.1" 200 1150 "-" "ELB-HealthChecker/2.0"  
172.31.30.6 - - [02/Feb/2018:06:25:06 +0000] "GET / HTTP/1.1" 200 82550 "-" "ELB-HealthChecker/2.0"  
172.31.10.17 - - [02/Feb/2018:06:25:07 +0000] "GET /favicon.ico HTTP/1.1" 200 1150 "-" "ELB-HealthChecker/2.0"  
172.31.28.216 - - [02/Feb/2018:06:25:08 +0000] "GET /favicon.ico HTTP/1.1" 200 1150 "-" "ELB-HealthChecker/2.0"  
172.31.4.52 - - [02/Feb/2018:06:25:08 +0000] "GET / HTTP/1.1" 200 82550 "-" "ELB-HealthChecker/2.0"  
172.31.29.66 - - [02/Feb/2018:06:25:09 +0000] "GET /favicon.ico HTTP/1.1" 200 1150 "-" "ELB-HealthChecker/2.0"  
172.31.30.6 - - [02/Feb/2018:06:25:12 +0000] "GET /c/oyo?id=IXCP3614&amount=16800&param1=-1 HTTP/1.1" 204 0 "[https://www.oyorooms.com/](https://www.oyorooms.com/)" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"  
172.31.12.194 - - [02/Feb/2018:06:25:15 +0000] "GET / HTTP/1.1" 200 82550 "-" "ELB-HealthChecker/2.0"  
172.31.0.51 - - [02/Feb/2018:06:25:15 +0000] "GET /favicon.ico HTTP/1.1" 200 1150 "-" "ELB-HealthChecker/2.0"  
172.31.20.253 - - [02/Feb/2018:06:25:22 +0000] "GET /favicon.ico HTTP/1.1" 200 1150 "-" "ELB-HealthChecker/2.0"

grok filter :  
grok {  
match =\> { "message" =\> "(?([0-9]{1,3}.)_[0-9]{1,3}) (?[-]) (?[-]) [(?[0-9]{1,2}/(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)/[0-9]{1,4}:[0-9]{1,2}:[0-9]{1,2}:[0-9]{1,2}\s+[0-9]{1,4})] "(?GET|PUT|POST|PATCH) (?/[a-zA-Z0-9]+/[a-zA-Z0-9]+)?(?([a-zA-Z0-9]+=[a-zA-Z0-9]+&)_[a-zA-Z0-9]+=[a-zA-Z0-9]+) HTTP/(?[0-9]+.[0-9])" (?[0-9]+) (?[0-9]+) "(?(http(s)?://)?(w._)?([a-zA-Z0-9]+.)+[a-zA-Z0-9]+(((?)?/?=?#?:?&?.?\s?[a-zA-Z0-9]?)_)?)" "(?.\*)""}  
}

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 25, 2018, 6:23am UTC](https://discuss.elastic.co/t/custom-pattern-in-grok-filter/129413/2 "2018-04-25T06:23:36Z")

</div>

This looks like a pretty typical apache/nginx log file, why are you building a pattern from scratch?

---

<div class="post-metadata">

**Author:** ![Akash\_Tanwar](https://avatars.discourse-cdn.com/v4/letter/a/3ab097/32.png) [@Akash\_Tanwar](https://discuss.elastic.co/u/Akash_Tanwar)\
**Post date:** [April 25, 2018, 6:44am UTC](https://discuss.elastic.co/t/custom-pattern-in-grok-filter/129413/3 "2018-04-25T06:44:41Z")

</div>

Learning purpose. In case there is a unique pattern in future, I would be able to handle it.  
I just want to know what is the syntax error. How can I include the custom pattern in grok filter?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 25, 2018, 6:45am UTC](https://discuss.elastic.co/t/custom-pattern-in-grok-filter/129413/4 "2018-04-25T06:45:38Z")

</div>

> [@Akash\_Tanwar](#):
>
> I just want to know what is the syntax error

You haven't provided the error so we can't really tell.

---

<div class="post-metadata">

**Author:** ![Akash\_Tanwar](https://avatars.discourse-cdn.com/v4/letter/a/3ab097/32.png) [@Akash\_Tanwar](https://discuss.elastic.co/u/Akash_Tanwar)\
**Post date:** [April 25, 2018, 6:47am UTC](https://discuss.elastic.co/t/custom-pattern-in-grok-filter/129413/5 "2018-04-25T06:47:55Z")

</div>

grok {  
match =\> { "message" =\> "(?([0-9]{1,3}.)_[0-9]{1,3}) (?[-]) (?[-]) [(?[0-9]{1,2}/(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)/[0-9]{1,4}:[0-9]{1,2}:[0-9]{1,2}:[0-9]{1,2}\s+[0-9]{1,4})] "(?GET|PUT|POST|PATCH) (?/[a-zA-Z0-9]+/[a-zA-Z0-9]+)?(?([a-zA-Z0-9]+=[a-zA-Z0-9]+&)_[a-zA-Z0-9]+=[a-zA-Z0-9]+) HTTP/(?[0-9]+.[0-9])" (?[0-9]+) (?[0-9]+) "(?(http(s)?://)?(w._)?([a-zA-Z0-9]+.)+[a-zA-Z0-9]+(((?)?/?=?#?:?&?.?\s?[a-zA-Z0-9]?)_)?)" "(?.\*)""}  
}

Error:  
[2018-04-25T12:17:33,989][ERROR][logstash.agent] Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of #, {, } at line 9, column 233 (byte 289) after filter {\n \tgrok {\n match =\> { "message" =\> "(?([0-9]{1,3}\.)\*[0-9]{1,3}) (?[-]) (?[-]) \[(?[0-9]{1,2}\/(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)\/[0-9]{1,4}\:[0-9]{1,2}\:[0-9]{1,2}\:[0-9]{1,2}\s\+[0-9]{1,4})] "", :backtrace=\>["/home/akash/logstash-6.2.3/logstash-core/lib/logstash/compiler.rb:42:in `compile_imperative'", "/home/akash/logstash-6.2.3/logstash-core/lib/logstash/compiler.rb:50:in`compile\_graph'", "/home/akash/logstash-6.2.3/logstash-core/lib/logstash/compiler.rb:12:in `block in compile_sources'", "org/jruby/RubyArray.java:2486:in`map'", "/home/akash/logstash-6.2.3/logstash-core/lib/logstash/compiler.rb:11:in `compile_sources'", "/home/akash/logstash-6.2.3/logstash-core/lib/logstash/pipeline.rb:51:in`initialize'", "/home/akash/logstash-6.2.3/logstash-core/lib/logstash/pipeline.rb:169:in `initialize'", "/home/akash/logstash-6.2.3/logstash-core/lib/logstash/pipeline_action/create.rb:40:in`execute'", "/home/akash/logstash-6.2.3/logstash-core/lib/logstash/agent.rb:315:in `block in converge_state'", "/home/akash/logstash-6.2.3/logstash-core/lib/logstash/agent.rb:141:in`with\_pipelines'", "/home/akash/logstash-6.2.3/logstash-core/lib/logstash/agent.rb:312:in `block in converge_state'", "org/jruby/RubyArray.java:1734:in`each'", "/home/akash/logstash-6.2.3/logstash-core/lib/logstash/agent.rb:299:in `converge_state'", "/home/akash/logstash-6.2.3/logstash-core/lib/logstash/agent.rb:166:in`block in converge\_state\_and\_update'", "/home/akash/logstash-6.2.3/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/home/akash/logstash-6.2.3/logstash-core/lib/logstash/agent.rb:164:in`converge\_state\_and\_update'", "/home/akash/logstash-6.2.3/logstash-core/lib/logstash/agent.rb:90:in `execute'", "/home/akash/logstash-6.2.3/logstash-core/lib/logstash/runner.rb:348:in`block in execute'", "/home/akash/logstash-6.2.3/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [April 25, 2018, 6:59am UTC](https://discuss.elastic.co/t/custom-pattern-in-grok-filter/129413/6 "2018-04-25T06:59:06Z")

</div>

You are using unescaped " within a text that is wrapped in ". LogStash thinks that the pattern ends there and doesn't know why more text is following. I think it should work if you wrap the pattern with ' instead.

---

<div class="post-metadata">

**Author:** ![JKhondhu](https://avatars.discourse-cdn.com/v4/letter/j/ed655f/32.png) [@JKhondhu](https://discuss.elastic.co/u/JKhondhu)\
**Post date:** [April 26, 2018, 8:40pm UTC](https://discuss.elastic.co/t/custom-pattern-in-grok-filter/129413/7 "2018-04-26T20:40:05Z")

</div>

@Akash_Tanwar  
For when you give up, we have some examples here to get you going: [https://github.com/logstash-plugins/logstash-patterns-core/tree/master/patterns](https://github.com/logstash-plugins/logstash-patterns-core/tree/master/patterns)

---

<div class="post-metadata">

**Author:** ![Akash\_Tanwar](https://avatars.discourse-cdn.com/v4/letter/a/3ab097/32.png) [@Akash\_Tanwar](https://discuss.elastic.co/u/Akash_Tanwar)\
**Post date:** [April 27, 2018, 6:27am UTC](https://discuss.elastic.co/t/custom-pattern-in-grok-filter/129413/8 "2018-04-27T06:27:08Z")

</div>

Thank You! I just put all the special characters like spaces, quotes and forward slashes with a '' and not as it is. And that solved my problem.

Thank you all for helping a newbie 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 25, 2018, 6:27am UTC](https://discuss.elastic.co/t/custom-pattern-in-grok-filter/129413/9 "2018-05-25T06:27:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.