# High CPU on random idle node **URL:** \ **Category:** Elasticsearch\ **Created:** [September 12, 2014, 10:10pm UTC](https://discuss.elastic.co/t/high-cpu-on-random-idle-node/19767 "2014-09-12T22:10:41Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Justin\_Lintz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/justin_lintz/32/1090_2.png) [@Justin\_Lintz](https://discuss.elastic.co/u/Justin_Lintz)\ **Post date:** [September 12, 2014, 10:10pm UTC](https://discuss.elastic.co/t/high-cpu-on-random-idle-node/19767/1 "2014-09-12T22:10:41Z")
Hi, We're deploying ES for logstash and I recently setup the cluster to migrate older indexes to "slower" nodes for longer term storage. The nodes are not being queried or doing any indexing but CPU is at 75% constantly. Here is output from hot\_threads , jstack and some settings $ java -version java version "1.7.0\_65" OpenJDK Runtime Environment (IcedTea 2.5.1) (7u65-2.5.1-4ubuntu1~0.12.04.2) OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode) hot threads: [https://gist.github.com/jlintz/3d965940284f1a7acf1e](https://gist.github.com/jlintz/3d965940284f1a7acf1e) settings: [https://gist.github.com/jlintz/c701496a3db26ff0a20e](https://gist.github.com/jlintz/c701496a3db26ff0a20e) jstack: [https://gist.github.com/jlintz/35924149197850e52931](https://gist.github.com/jlintz/35924149197850e52931) I just realized while posting this my index buffers are set too high for these nodes since they arent doing indexing, but just in case that's not the issue, I'll post anyway. I'll report back if issue is still present -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6b987879-3997-4c5f-9753-14cab0dae545%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6b987879-3997-4c5f-9753-14cab0dae545%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [September 12, 2014, 10:36pm UTC](https://discuss.elastic.co/t/high-cpu-on-random-idle-node/19767/2 "2014-09-12T22:36:17Z")
Looks like you have a monitoring tool running and it got stuck in the node stats call while traversing a number of shards/segments. How many shards/segments are in your migration? It seems to be very active. Maybe the bloom filter format conversion is expensive but I am not sure. Jörg On Sat, Sep 13, 2014 at 12:10 AM, Justin Lintz [jlintz@gmail.com](mailto:jlintz@gmail.com) wrote: > Hi, > > We're deploying ES for logstash and I recently setup the cluster to > migrate older indexes to "slower" nodes for longer term storage. The nodes > are not being queried or doing any indexing but CPU is at 75% constantly. > Here is output from hot\_threads , jstack and some settings > > $ java -version > java version "1.7.0\_65" > OpenJDK Runtime Environment (IcedTea 2.5.1) (7u65-2.5.1-4ubuntu1~0.12.04.2) > OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode) > > hot threads: [hot\_threads · GitHub](https://gist.github.com/jlintz/3d965940284f1a7acf1e) > settings: [ES settings · GitHub](https://gist.github.com/jlintz/c701496a3db26ff0a20e) > jstack: [jstack · GitHub](https://gist.github.com/jlintz/35924149197850e52931) > > I just realized while posting this my index buffers are set too high for > these nodes since they arent doing indexing, but just in case that's not > the issue, I'll post anyway. I'll report back if issue is still present > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/6b987879-3997-4c5f-9753-14cab0dae545%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6b987879-3997-4c5f-9753-14cab0dae545%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/6b987879-3997-4c5f-9753-14cab0dae545%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/6b987879-3997-4c5f-9753-14cab0dae545%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGCJvGZU-tjQaCxC57MgXcy63TR7mDE1reHwAKzyRfoMg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGCJvGZU-tjQaCxC57MgXcy63TR7mDE1reHwAKzyRfoMg%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Justin\_Lintz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/justin_lintz/32/1090_2.png) [@Justin\_Lintz](https://discuss.elastic.co/u/Justin_Lintz)\ **Post date:** [September 13, 2014, 12:01am UTC](https://discuss.elastic.co/t/high-cpu-on-random-idle-node/19767/3 "2014-09-13T00:01:51Z")
Thanks, it ended up being the Ganglia plugin that was causing crazy CPU consumption. I've disabled it since we'll end up buying Marvel once we've fully deployed. On Friday, September 12, 2014 6:36:27 PM UTC-4, Jörg Prante wrote: > Looks like you have a monitoring tool running and it got stuck in the node > stats call while traversing a number of shards/segments. > > How many shards/segments are in your migration? It seems to be very active. > > Maybe the bloom filter format conversion is expensive but I am not sure. > > Jörg > > On Sat, Sep 13, 2014 at 12:10 AM, Justin Lintz \<[jli...@gmail.com](mailto:jli...@gmail.com) > \\> wrote: > > > Hi, > > > > We're deploying ES for logstash and I recently setup the cluster to > > migrate older indexes to "slower" nodes for longer term storage. The nodes > > are not being queried or doing any indexing but CPU is at 75% constantly. > > Here is output from hot\_threads , jstack and some settings > > > > $ java -version > > java version "1.7.0\_65" > > OpenJDK Runtime Environment (IcedTea 2.5.1) > > (7u65-2.5.1-4ubuntu1~0.12.04.2) > > OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode) > > > > hot threads: [hot\_threads · GitHub](https://gist.github.com/jlintz/3d965940284f1a7acf1e) > > settings: [ES settings · GitHub](https://gist.github.com/jlintz/c701496a3db26ff0a20e) > > jstack: [jstack · GitHub](https://gist.github.com/jlintz/35924149197850e52931) > > > > I just realized while posting this my index buffers are set too high for > > these nodes since they arent doing indexing, but just in case that's not > > the issue, I'll post anyway. I'll report back if issue is still present > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/6b987879-3997-4c5f-9753-14cab0dae545%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6b987879-3997-4c5f-9753-14cab0dae545%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/6b987879-3997-4c5f-9753-14cab0dae545%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/6b987879-3997-4c5f-9753-14cab0dae545%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/463d5e56-57da-4680-ab39-65dc484daa11%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/463d5e56-57da-4680-ab39-65dc484daa11%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:02am UTC](https://discuss.elastic.co/t/high-cpu-on-random-idle-node/19767/4 "2017-07-06T01:02:38Z")