# Very low CPU/Mem usage in logstash server
**URL:** https://discuss.elastic.co/t/very-low-cpu-mem-usage-in-logstash-server/61667
**Category:** Logstash
**Created:** [September 28, 2016, 7:31am UTC](https://discuss.elastic.co/t/very-low-cpu-mem-usage-in-logstash-server/61667 "2016-09-28T07:31:31Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@caoping](https://discuss.elastic.co/u/caoping)
#### Post date: [September 28, 2016, 7:31am UTC](https://discuss.elastic.co/t/very-low-cpu-mem-usage-in-logstash-server/61667/1 "2016-09-28T07:31:31Z")
Hi,
My logstash server has 4 CPUs and 16g Mem, and 4 pipeline-workers (It's said the number of pipeline-worker should be the number of CPUs. ). However, after running more than 20 hours, I found it said:
> Beats input: The circuit breaker has detected a slowdown or stall in the pipeline...
Then I changed 'congestion\_threshold' and 'timeout' to a much larger number by referring to the solution on the Internet, and restarted logstash. Unfortunately, also after about 20+ hours, the same issue occurred.
I checked the CPU/Mem usage with **top** command, and found CPU was about 1%, and Mem was about 3%. Very low usage.
I'm not sure why the pipeline was blocked under such low CPU/Mem usage. And what else can I do to avoid that issue? Any help will be appreciated.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [September 28, 2016, 7:42am UTC](https://discuss.elastic.co/t/very-low-cpu-mem-usage-in-logstash-server/61667/2 "2016-09-28T07:42:05Z")
The recommended settings for these parameters have changed over the last few releases. Which version of Logstash are you using? What does your config look like?
---
### Author:  [@caoping](https://discuss.elastic.co/u/caoping)
#### Post date: [September 28, 2016, 9:20am UTC](https://discuss.elastic.co/t/very-low-cpu-mem-usage-in-logstash-server/61667/3 "2016-09-28T09:20:59Z")
Sorry, I forgot to add logstash version information. The logstash version is 2.3.4
---
### Author:  [@caoping](https://discuss.elastic.co/u/caoping)
#### Post date: [October 11, 2016, 8:41am UTC](https://discuss.elastic.co/t/very-low-cpu-mem-usage-in-logstash-server/61667/4 "2016-10-11T08:41:45Z")
Hi,
My logstash config is:
>
input {
beats {
port =\> 5044
ssl =\> true
ssl\_certificate =\> "/etc/pki/tls/certs/logstash.crt"
ssl\_key =\> "/etc/pki/tls/private/logstash.key"
}
}
>
filter {
grok {
match =\> {
"message" =\> [
"[(?(%{MONTHNUM}/%{MONTHDAY}/%{YEAR})\s+%{TIME}\s+%{WORD})]\s+%{BASE16NUM:ThreadID}\s+(?([\w|\S]+))\s+%{WORD:LogLevel}\s+(?[\w|\W]_(?(SR[A-Za-z\d]_[\d]+))[\W]+[\w|\W]_)",
"[(?(%{MONTHNUM}/%{MONTHDAY}/%{YEAR})\s+%{TIME}\s+%{WORD})]\s+%{BASE16NUM:ThreadID}\s+(?([\w|\S]+))\s+%{WORD:LogLevel}\s+(?[\w|\W]_(\n)+(?(SR[A-Za-z\d]_[\d]+))(\n)+[\w|\W]_)"
]
}
remove\_field =\> ["message"]
}
if "\_grokparsefailure" in [tags] {
grok {
match =\> ["message", "[(?(%{MONTHNUM}/%{MONTHDAY}/%{YEAR})\s+%{TIME}\s+%{WORD})]\s+%{BASE16NUM:ThreadID}\s+(?([\w|\S]+))\s+%{WORD:LogLevel}\s+(?[\w|\W]_)"]
remove\_field =\> ["message"]
remove\_tag =\> ["\_grokparsefailure"]
add\_field =\> {
SRNumber =\> "-"
}
}
}
if "\_grokparsefailure" in [tags] {
grok {
match =\> ["message", "[(?(%{MONTHNUM}/%{MONTHDAY}/%{YEAR})\s+%{TIME}\s+%{WORD})]\s+%{BASE16NUM:ThreadID}\s+%{WORD:LogLevel}\s+(?[\w|\W]_)"]
remove\_field =\> ["message"]
remove\_tag =\> ["\_grokparsefailure"]
add\_field =\> {
SRNumber =\> "-"
LogSOurce =\> "-"
}
}
}
if "\_grokparsefailure" in [tags] {
grok {
match =\> ["message", "(?[\w|\W]+)"]
remove\_field =\> ["message"]
remove\_tag =\> ["\_grokparsefailure"]
add\_tag =\> ["ignore"]
add\_field =\> {
LogSource =\> "-"
LogLevel =\> "-"
SRNumber =\> "-"
LogTime =\> "-"
ThreadID =\> "-"
}
}
}
if "SWIS" in [fields][ServerType] {
date {
match =\> ["Logtime", "M/d/yy HH:mm:ss:SSS z"]
timezone =\> "GMT"
}
} else {
date {
match =\> ["Logtime", "M/d/yy HH:mm:ss:SSS z"]
timezone =\> "UTC"
}
}
}
output {
elasticsearch {
hosts =\> "IP"
index =\> "logstash-site-%{+YYYY.MM.dd}"
flush\_size =\> 50
}
}
After increasing the number of pipeline workers from 4 to 8, logstash had been working well for about 10 days. It's a good progress compared to before it only worked well for less than 1 days.
Is anything wrong with my configuration? If yes, any suggestions for that? This issue has bothered my for a long time, I'll be appreciated for your help.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [October 13, 2016, 5:28am UTC](https://discuss.elastic.co/t/very-low-cpu-mem-usage-in-logstash-server/61667/5 "2016-10-13T05:28:14Z")
> [@caoping](#):
>
> flush\_size =\> 50
Why have you specified such a small value? I would stick with the default values as this could have an impact on performance as it will require additional round trips to Elasticsearch.
Although I don't know exactly what your data looks like and what proportion of data that is matched by the various patterns, it looks like all patterns start with the same sequence: `[(?(%{MONTHNUM}/%{MONTHDAY}/%{YEAR})\s+%{TIME}\s+%{WORD})]\s+%{BASE16NUM:ThreadID}\s+`
It may be more efficient to capture this in one grok filter and use a GREEDYDATA to capture the rest of the message into a separate variable that can then be matched against the various scenarios.
---
### Author:  [@caoping](https://discuss.elastic.co/u/caoping)
#### Post date: [October 17, 2016, 2:28pm UTC](https://discuss.elastic.co/t/very-low-cpu-mem-usage-in-logstash-server/61667/6 "2016-10-17T14:28:49Z")
Yes, you're right. The flush\_size is too small. I didn't notice that before your reminder. Thank a lot.
Your suggestion on grok also makes sense, but after I update the grok filter, there seems no improvements on logstash performance. It met the pipeline slowdown issue after several hours running. I may need to revise my grok configuration.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:33am UTC](https://discuss.elastic.co/t/very-low-cpu-mem-usage-in-logstash-server/61667/7 "2017-07-06T04:33:55Z")