# Grok performance problem when parsing dots, dashs, underscore

**URL:** <https://discuss.elastic.co/t/grok-performance-problem-when-parsing-dots-dashs-underscore/102985>\
**Category:** Logstash\
**Created:** [October 6, 2017, 9:45am UTC](https://discuss.elastic.co/t/grok-performance-problem-when-parsing-dots-dashs-underscore/102985 "2017-10-06T09:45:38Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![simon.t](https://avatars.discourse-cdn.com/v4/letter/s/4bbf92/32.png) [@simon.t](https://discuss.elastic.co/u/simon.t)\
**Post date:** [October 6, 2017, 9:45am UTC](https://discuss.elastic.co/t/grok-performance-problem-when-parsing-dots-dashs-underscore/102985/1 "2017-10-06T09:45:38Z")

</div>

Hi all,  
I'm issuing a performance problem with grok filter.  
I use filebeat 5.4.6 to send log file event to logstash 5.4.6.  
I made a very simple grok filter in logstash to extract path and filename of the log file from "source" field from filebeat :  
grok {  
match =\> { "source" =\> "%{UNIXPATH:[filepath]}/%{NOTSPACE:[filename]}" }  
}

It works very well with a lot of filename but the filter is very slow when there is many dots, dashs, underscore in the filename.  
Example : /var/log/nginx/mynginx01access.log -\> very fast  
/var/log/nginx/my\_nginx-01.access.log -\> very slow and CPU costly

I try many pattern to replace %{NOTSPACE } whith %{DATA}, %{GREEDYDATA}... whitout any result. The CPU loads for the filter seems to be an exponential of the number of (.,-,_) in the filename.  
If you replace (.,-,_) whith other special charater (#,$,^,space...), it's fast again.

I don't know how to fix this problem, because I try every possible pattern.

Help would be very appreciated.

Simon

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [October 6, 2017, 10:20am UTC](https://discuss.elastic.co/t/grok-performance-problem-when-parsing-dots-dashs-underscore/102985/2 "2017-10-06T10:20:10Z")

</div>

Read this [https://www.elastic.co/blog/do-you-grok-grok](https://www.elastic.co/blog/do-you-grok-grok)

then after that try:  
`^%{UNIXPATH:[filepath]}/%{JAVAFILE:[filename]}$`

---

<div class="post-metadata">

**Author:** ![simon.t](https://avatars.discourse-cdn.com/v4/letter/s/4bbf92/32.png) [@simon.t](https://discuss.elastic.co/u/simon.t)\
**Post date:** [October 6, 2017, 11:09am UTC](https://discuss.elastic.co/t/grok-performance-problem-when-parsing-dots-dashs-underscore/102985/3 "2017-10-06T11:09:05Z")

</div>

Thank you for your response.  
First, I made a mistake on the versions of filebeat and Logstash, i work on the latest 5.6.2 on CentOS 7u3.  
I try to change the match expression with your hint but it didn't solve the problem. I reproduce the same bad execution time.

---

<div class="post-metadata">

**Author:** ![simon.t](https://avatars.discourse-cdn.com/v4/letter/s/4bbf92/32.png) [@simon.t](https://discuss.elastic.co/u/simon.t)\
**Post date:** [October 6, 2017, 11:19am UTC](https://discuss.elastic.co/t/grok-performance-problem-when-parsing-dots-dashs-underscore/102985/4 "2017-10-06T11:19:31Z")

</div>

I made a few more tests and I solved the problem by changing UNIXPATH pattern by DATA or GREEDYDATA.  
grok {  
match =\> { "source" =\> "^%{DATA:[fields][filepath]}/%{JAVAFILE:[fields][filename]}$" }  
}

I don't understand how the matching on UNIXPATH pattern is dependant on the format of the last part of string after the "/".

Thank you for your first response.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 3, 2017, 11:19am UTC](https://discuss.elastic.co/t/grok-performance-problem-when-parsing-dots-dashs-underscore/102985/5 "2017-11-03T11:19:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.