# Unexpected high CPU / IO loading **URL:** \ **Category:** Elasticsearch\ **Created:** [March 29, 2015, 2:28am UTC](https://discuss.elastic.co/t/unexpected-high-cpu-io-loading/22966 "2015-03-29T02:28:03Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Felix\_Ng](https://avatars.discourse-cdn.com/v4/letter/f/b77776/32.png) [@Felix\_Ng](https://discuss.elastic.co/u/Felix_Ng)\ **Post date:** [March 29, 2015, 2:28am UTC](https://discuss.elastic.co/t/unexpected-high-cpu-io-loading/22966/1 "2015-03-29T02:28:03Z")
My cluster have unexpected high loading even there is no query / indexing recently. I suspect it is the segment merging. 1. How can I tell if it is becuase of merging? I tried to look into the HotThread... but I can't really understand the meaning. 2. I tried to turn off merging completely. That means set "indices.store.throttle.type" to none. It seems there is some improvement, but STILL swa some unexpected high loading even when idle. 3. To compensate turning off auto segment merge, I execute "[http://localhost:9200/\_all/\_optimize?max\_num\_segments=6](http://localhost:9200/_all/_optimize?max_num_segments=6)" every night. Is this an appropriate workaround? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/634edf53-24b1-4ac1-9f35-b8ae1a3a3e91%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/634edf53-24b1-4ac1-9f35-b8ae1a3a3e91%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [March 29, 2015, 7:48am UTC](https://discuss.elastic.co/t/unexpected-high-cpu-io-loading/22966/2 "2015-03-29T07:48:37Z")
How high? ES will do things int he background (mostly around lucene merging), but is it causing a problem? On 29 March 2015 at 13:28, Felix Ng [felixng2k@gmail.com](mailto:felixng2k@gmail.com) wrote: > My cluster have unexpected high loading even there is no query / indexing > recently. I suspect it is the segment merging. > > 1. How can I tell if it is becuase of merging? I tried to look into the > HotThread... but I can't really understand the meaning. > 2. I tried to turn off merging completely. That means set > "indices.store.throttle.type" to none. It seems there is some improvement, > but STILL swa some unexpected high loading even when idle. > 3. To compensate turning off auto segment merge, I execute " > [http://localhost:9200/\_all/\_optimize?max\_num\_segments=6](http://localhost:9200/_all/_optimize?max_num_segments=6)" every night. Is > this an appropriate workaround? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/634edf53-24b1-4ac1-9f35-b8ae1a3a3e91%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/634edf53-24b1-4ac1-9f35-b8ae1a3a3e91%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/634edf53-24b1-4ac1-9f35-b8ae1a3a3e91%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/634edf53-24b1-4ac1-9f35-b8ae1a3a3e91%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_rY26mFBett11Xxm92GFq4m%3DtuOTcMagV9NEMQr7ZcJw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_rY26mFBett11Xxm92GFq4m%3DtuOTcMagV9NEMQr7ZcJw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Felix\_Ng](https://avatars.discourse-cdn.com/v4/letter/f/b77776/32.png) [@Felix\_Ng](https://discuss.elastic.co/u/Felix_Ng)\ **Post date:** [March 29, 2015, 11:32am UTC](https://discuss.elastic.co/t/unexpected-high-cpu-io-loading/22966/3 "2015-03-29T11:32:22Z")
Average loading around 30 - 40. I had 9 nodes, all Intel i7, average loading stay under 4 when ES is 'normal'. I had attached 3 copies of HotThread results, they were captured within 20 mins. The "Lucene Merge Thread" appears on 2 to 3 nodes, using 50-60% CPU, but as i mentioned above, the system load-average are over 30! On Sunday, March 29, 2015 at 3:49:05 PM UTC+8, Mark Walkom wrote: > How high? > > ES will do things int he background (mostly around lucene merging), but is > it causing a problem? > > On 29 March 2015 at 13:28, Felix Ng \<[feli...@gmail.com](mailto:feli...@gmail.com) \\> > wrote: > > > My cluster have unexpected high loading even there is no query / indexing > > recently. I suspect it is the segment merging. > > > > 1. How can I tell if it is becuase of merging? I tried to look into the > > HotThread... but I can't really understand the meaning. > > 2. I tried to turn off merging completely. That means set > > "indices.store.throttle.type" to none. It seems there is some improvement, > > but STILL swa some unexpected high loading even when idle. > > 3. To compensate turning off auto segment merge, I execute " > > [http://localhost:9200/\_all/\_optimize?max\_num\_segments=6](http://localhost:9200/_all/_optimize?max_num_segments=6)" every night. > > Is this an appropriate workaround? > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/634edf53-24b1-4ac1-9f35-b8ae1a3a3e91%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/634edf53-24b1-4ac1-9f35-b8ae1a3a3e91%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/634edf53-24b1-4ac1-9f35-b8ae1a3a3e91%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/634edf53-24b1-4ac1-9f35-b8ae1a3a3e91%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/4660db9a-8733-4ae8-962c-735df932bbad%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4660db9a-8733-4ae8-962c-735df932bbad%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [March 29, 2015, 2:48pm UTC](https://discuss.elastic.co/t/unexpected-high-cpu-io-loading/22966/4 "2015-03-29T14:48:47Z")
1. If you are not sure about merging, you should look for other reasons for high load. Identify the processes with high activity. Check if your storage I/O system can keep up. 2. You can not turn off merging. With indices.store.throttle.type: none you diasble throttling. 3. Optimizing by manual operation is always second best and not required every night. Jörg On Sun, Mar 29, 2015 at 4:28 AM, Felix Ng [felixng2k@gmail.com](mailto:felixng2k@gmail.com) wrote: > My cluster have unexpected high loading even there is no query / indexing > recently. I suspect it is the segment merging. > > 1. How can I tell if it is becuase of merging? I tried to look into the > HotThread... but I can't really understand the meaning. > 2. I tried to turn off merging completely. That means set > "indices.store.throttle.type" to none. It seems there is some improvement, > but STILL swa some unexpected high loading even when idle. > 3. To compensate turning off auto segment merge, I execute " > [http://localhost:9200/\_all/\_optimize?max\_num\_segments=6](http://localhost:9200/_all/_optimize?max_num_segments=6)" every night. Is > this an appropriate workaround? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/634edf53-24b1-4ac1-9f35-b8ae1a3a3e91%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/634edf53-24b1-4ac1-9f35-b8ae1a3a3e91%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/634edf53-24b1-4ac1-9f35-b8ae1a3a3e91%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/634edf53-24b1-4ac1-9f35-b8ae1a3a3e91%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEM4OTm1wUe%3Dfcy3N%3D0wgWp6hhkz%3D66tMUL%3DfMKWJzTvw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEM4OTm1wUe%3Dfcy3N%3D0wgWp6hhkz%3D66tMUL%3DfMKWJzTvw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:23am UTC](https://discuss.elastic.co/t/unexpected-high-cpu-io-loading/22966/5 "2017-07-06T00:23:11Z")