# Abnormally CPU utilization by Logstash

**URL:** <https://discuss.elastic.co/t/abnormally-cpu-utilization-by-logstash/203386>\
**Category:** Logstash\
**Created:** [October 14, 2019, 6:26am UTC](https://discuss.elastic.co/t/abnormally-cpu-utilization-by-logstash/203386 "2019-10-14T06:26:27Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![emilio](https://avatars.discourse-cdn.com/v4/letter/e/f475e1/32.png) [@emilio](https://discuss.elastic.co/u/emilio)\
**Post date:** [October 14, 2019, 6:26am UTC](https://discuss.elastic.co/t/abnormally-cpu-utilization-by-logstash/203386/1 "2019-10-14T06:26:27Z")

</div>

Dear all,

I have just one grok filter in my logstash and it horrible utilizes CPU up to 1600%

Could you, please help me to detect bottleneck in my filter and fix it?

Thank you in advance.

That's my filter:

filter {

if [fields][type] == "test\_st" {  
grok {  
match =\> { "message" =\> ["(?\<action\_date\>%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{TIME}.(?\d{0,3})) [(?\<session\_id\>%{WORD:username}  
?[^]]_)] .HOST[=]%{HOSTNAME:db\_name}((.|\n)_)).[\s\n]_(?\<sql\_command\>[(\w+]_ ((.|\n)_))[\s\n]_{(?\<d\_time\>[^}]_)}",  
"(?\<action\_date\>%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{TIME}.(?\d{0,3})) [(?\<session\_id\>%{WORD:username}?[^]]_)] ._@(?\<db\_name\>._)[\s\n]_(?\<sql\_command\>[(\w+]_ ((.|\n)_))[\s\n]_{(?\<d\_time\>[^}]\*)}"]}  
}  
date {  
match =\> ["action\_date","yyyy-MM-dd HH:mm:ss.SSS"]  
target =\> "@timestamp"  
}  
}

else if [fields][type] == "test\_log" {  
grok {  
match =\> { "message" =\> ["%{DATESTAMP:action\_date}\s+[%{LOGLEVEL:log\_level}]\s+{(?\<app\_log\>[^}]_)}\s+[(?\<session\_id\>%{WORD:username}?[^]]_)]\s+-\s+(?\<command\_response\>(((.|\n)_)?(?=^\d[^\d])((.|\n)_))|((.|\n))\*)"]}  
}

date {  
match =\> ["action\_date","dd-MM-yyyy HH:mm:ss"]  
target =\> "@timestamp"  
}  
}

else if [fields][type] == "test\_d" {  
grok {  
match =\> { "message" =\> ["%{DATESTAMP:action\_date} [%{LOGLEVEL:log\_level}] {(?\<app\_log\>[^}]_)} [(?\<session\_id\>%{WORD:username}?[^]]_)] -._: (?._),User:._,Start date2:(?\<start\_date\>._),End date1:(?\<end\_date\>.\*),"]}  
}  
date {  
match =\> ["action\_date","dd-MM-yyyy HH:mm:ss"]  
target =\> "@timestamp"  
}  
}

else if [fields][type] == "test\_error" {  
grok {  
match =\> { "message" =\> ["%{DATESTAMP:action\_date} [%{LOGLEVEL:log\_level}] {(?\<app\_log\>[^}]_)} [(?\<session\_id\>%{WORD:username}?[^]]_)] - [._] (?((.|\n))_)"]}  
}  
date {  
match =\> ["action\_date","dd-MM-yyyy HH:mm:ss"]  
target =\> "@timestamp"  
}  
}

```
else if [fields][type] == "robin" {
 grok {
 match => { "message" => ["(?<action_date>%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{TIME},(?<milliseconds>\d{0,3}))\s*%{LOGLEVEL:log_level}\s\[(?<session_id>(?<sms_id>\d+)?-(?<msisdn>\d+)?-[^\]]*)\]\s*\((?<app_log>[^\)]*)\)\s*(?<command_response>((.|\n))*)"]}
 }
 date {
   match => ["action_date","yyyy-MM-dd HH:mm:ss,SSS"]
   target => "@timestamp"
  }
 }

```

}

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 14, 2019, 6:49am UTC](https://discuss.elastic.co/t/abnormally-cpu-utilization-by-logstash/203386/2 "2019-10-14T06:49:59Z")

</div>

Your grok expressions are very hard to read as you have not formatted the config correctly. I suspect you may have inefficient grok expressions and that might cause the high CPU but without proper formatting it is hard to use well as e.g. \* turns text into italic rather than being visible.

---

<div class="post-metadata">

**Author:** ![emilio](https://avatars.discourse-cdn.com/v4/letter/e/f475e1/32.png) [@emilio](https://discuss.elastic.co/u/emilio)\
**Post date:** [October 25, 2019, 7:20am UTC](https://discuss.elastic.co/t/abnormally-cpu-utilization-by-logstash/203386/3 "2019-10-25T07:20:45Z")

</div>

Could you, please tell me what is wrong with this expression?

["^%{DATESTAMP:action\_date}\s+[%{LOGLEVEL:log\_level}]\s+{(?\<app\_log\>[^}]_)}\s+[(?\<session\_id\>%{WORD:username}?[^]]_)]\s+-\s+(?\<command\_response\>((.|\n))\*)$"]

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [October 25, 2019, 12:24pm UTC](https://discuss.elastic.co/t/abnormally-cpu-utilization-by-logstash/203386/4 "2019-10-25T12:24:37Z")

</div>

You need to format your post so that characters are not consumed as [markdown](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet). Notice how you expression turns italic half way through. That is because characters from your regexp are being interpreted as markdown.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 22, 2019, 12:24pm UTC](https://discuss.elastic.co/t/abnormally-cpu-utilization-by-logstash/203386/5 "2019-11-22T12:24:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.