# Logstash at 100% CPU, slow to process Redis queue to Elasticsearch

**URL:** <https://discuss.elastic.co/t/logstash-at-100-cpu-slow-to-process-redis-queue-to-elasticsearch/49686>\
**Category:** Logstash\
**Created:** [May 10, 2016, 4:30pm UTC](https://discuss.elastic.co/t/logstash-at-100-cpu-slow-to-process-redis-queue-to-elasticsearch/49686 "2016-05-10T16:30:55Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![RCMMike](https://avatars.discourse-cdn.com/v4/letter/r/35a633/32.png) [@RCMMike](https://discuss.elastic.co/u/RCMMike)\
**Post date:** [May 10, 2016, 4:30pm UTC](https://discuss.elastic.co/t/logstash-at-100-cpu-slow-to-process-redis-queue-to-elasticsearch/49686/1 "2016-05-10T16:30:55Z")

</div>

Hello! I have a Logstash instance that is reading data from a Redis queue, processing it via some grok filters, and then outputting to Elasticsearch.

For a day or two, this was working properly - decent CPU but not overloaded and processing quickly. However, starting today, it is now using 100% CPU (on 24 cores!) and is processing records at a snail's pace. It seems to read in about 4K records, process them, and then sit there at high CPU until reading more records a couple of minutes later.

I've tried to modify the ES\_HEAP\_SIZE and pipeline threads with no effect shown. I'm not seeing anything crazy in the logs (or really, much at all) and the server seems happy otherwise.

`input { redis { host => "127.0.0.1" data_type => "list" key => "filebeat" add_field => { "beattype" => "filebeat" } } } filter { mutate { rename => { "@metadata" => "metadata" } } } filter { if [type] == "referlog" { grok { match => { "message" => "%{REFERLOGENTRY}" } } date { locale => "en" timezone => "America/Los_Angeles" match => ["timestamp", "YYYY-MM-dd HH:mm:ss"] } mutate { gsub => ["txid", "\"", ""] gsub => ["txid", " ", ""] gsub => ["email", "\"", ""] gsub => ["referrer", "\"", ""] gsub => ["useragent", "\"", ""] } if [useragent] != "-" and [useragent] != "" { useragent { add_tag => ["UA"] source => "useragent" } } if "UA" in [tags] { if [device] == "Other" { mutate { remove_field => "device" } } if [name] == "Other" { mutate { remove_field => "name" } } if [os] == "Other" { mutate { remove_field => "os" } } } geoip { source => "clientip" target => "geoip" database => "/etc/logstash/GeoIP.dat" add_field => ["[geoip][coordinates]", "%{[geoip][longitude]}" ] add_field => ["[geoip][coordinates]", "%{[geoip][latitude]}" ] } mutate { convert => ["[geoip][coordinates]", "float"] } } } output { elasticsearch { hosts => ["192.168.0.2:9200"] sniffing => false manage_template => false index => "%{[beattype]}-%{[type]}-%{+YYYY.MM}" document_type => "%{[type]}" } }`

Any idea where I can start looking? Again, this was working fine with no changes a day or two ago, and just started going silly today.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 11, 2016, 11:33pm UTC](https://discuss.elastic.co/t/logstash-at-100-cpu-slow-to-process-redis-queue-to-elasticsearch/49686/2 "2016-05-11T23:33:16Z")

</div>

What version of things are you running?

---

<div class="post-metadata">

**Author:** ![michbsd](https://avatars.discourse-cdn.com/v4/letter/m/b77776/32.png) [@michbsd](https://discuss.elastic.co/u/michbsd)\
**Post date:** [May 29, 2016, 8:17pm UTC](https://discuss.elastic.co/t/logstash-at-100-cpu-slow-to-process-redis-queue-to-elasticsearch/49686/3 "2016-05-29T20:17:19Z")

</div>

Did you find a solution to this? I am seeing something very similar.

Thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:55am UTC](https://discuss.elastic.co/t/logstash-at-100-cpu-slow-to-process-redis-queue-to-elasticsearch/49686/4 "2017-07-06T04:55:39Z")

</div>