# Nodes spend all time in RamUsageEstimator after upgrade to 0.90.11 **URL:** \ **Category:** Elasticsearch\ **Created:** [February 25, 2014, 3:13pm UTC](https://discuss.elastic.co/t/nodes-spend-all-time-in-ramusageestimator-after-upgrade-to-0-90-11/16012 "2014-02-25T15:13:43Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Benoit](https://avatars.discourse-cdn.com/v4/letter/b/5daacb/32.png) [@Benoit](https://discuss.elastic.co/u/Benoit)\ **Post date:** [February 25, 2014, 3:13pm UTC](https://discuss.elastic.co/t/nodes-spend-all-time-in-ramusageestimator-after-upgrade-to-0-90-11/16012/1 "2014-02-25T15:13:43Z")
Hi all, After an upgrade from 0.90.3 to 0.90.11 I see my nodes spending lots of time doing RamUsageEstimator and I worry about the stability of my cluster ... Beginning of the hot\_thread response: 101.9% (509.3ms out of 500ms) cpu usage by thread 'elasticsearch[sissor1][management][T#1]' 10/10 snapshots sharing following 19 elements org.apache.lucene.util.RamUsageEstimator$IdentityHashSet.expandAndRehash(RamUsageEstimator.java:747) org.apache.lucene.util.RamUsageEstimator$IdentityHashSet.add(RamUsageEstimator.java:678) org.apache.lucene.util.RamUsageEstimator.measureObjectSize(RamUsageEstimator.java:437) org.apache.lucene.util.RamUsageEstimator.sizeOf(RamUsageEstimator.java:350) org.apache.lucene.codecs.lucene3x.Lucene3xFields.ramBytesUsed(Lucene3xFields.java:1080) org.apache.lucene.index.SegmentCoreReaders.ramBytesUsed(SegmentCoreReaders.java:195) org.apache.lucene.index.SegmentReader.ramBytesUsed(SegmentReader.java:558) org.elasticsearch.index.engine.robin.RobinEngine.getReaderRamBytesUsed(RobinEngine.java:1180) org.elasticsearch.index.engine.robin.RobinEng0 0 21052 0 0:00:01 0:00:01 --:--:-- 21092 ine.segmentsStats(RobinEngine.java:1192) org.elasticsearch.index.shard.service.InternalIndexShard.segmentStats(InternalIndexShard.java:514) org.elasticsearch.action.admin.indices.stats.CommonStats.(CommonStats.java:154) org.elasticsearch.indices.InternalIndicesService.stats(InternalIndicesService.java:211) org.elasticsearch.node.service.NodeService.stats(NodeService.java:165) org.elasticsearch.action.admin.cluster.node.stats.TransportNodesStatsAction.nodeOperation(TransportNodesStatsAction.java:100) org.elasticsearch.action.admin.cluster.node.stats.TransportNodesStatsAction.nodeOperation(TransportNodesStatsAction.java:43) org.elasticsearch.action.support.nodes.TransportNodesOperationAction$AsyncAction$2.run(TransportNodesOperationAction.java:146) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:724) 101.7% (508.3ms out of 500ms) cpu usage by thread 'elasticsearch[sissor1][management][T#2]' 10/10 snapshots sharing following 18 elements java.lang.reflect.Array.get(Native Method) org.apache.lucene.util.RamUsageEstimator.measureObjectSize(RamUsageEstimator.java:456) org.apache.lucene.util.RamUsageEstimator.sizeOf(RamUsageEstimator.java:350) org.apache.lucene.codecs.lucene3x.Lucene3xFields.ramBytesUsed(Lucene3xFields.java:1080) org.apache.lucene.index.SegmentCoreReaders.ramBytesUsed(SegmentCoreReaders.java:195) org.apache.lucene.index.SegmentReader.ramBytesUsed(SegmentReader.java:558) org.elasticsearch.index.engine.robin.RobinEngine.getReaderRamBytesUsed(RobinEngine.java:1180) org.elasticsearch.index.engine.robin.RobinEngine.segmentsStats(RobinEngine.java:1192) org.elasticsearch.index.shard.service.InternalIndexShard.segmentStats(InternalIndexShard.java:514) org.elasticsearch.action.admin.indices.stats.CommonStats.(CommonStats.java:154) org.elasticsearch.indices.InternalIndicesService.stats(InternalIndicesService.java:211) org.elasticsearch.node.service.NodeService.stats(NodeService.java:165) org.elasticsearch.action.admin.cluster.node.stats.TransportNodesStatsAction.nodeOperation(TransportNodesStatsAction.java:100) org.elasticsearch.action.admin.cluster.node.stats.TransportNodesStatsAction.nodeOperation(TransportNodesStatsAction.java:43) org.elasticsearch.action.support.nodes.TransportNodesOperationAction$AsyncAction$2.run(TransportNodesOperationAction.java:146) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:724) I've seen this message [https://groups.google.com/forum/#!searchin/elasticsearch/RamUsageEstimator/elasticsearch/7mrDhqe6LEo/Vgv4UtvvvEIJ](https://groups.google.com/forum/#!searchin/elasticsearch/RamUsageEstimator/elasticsearch/7mrDhqe6LEo/Vgv4UtvvvEIJ) But there is no conclusion, can I do somethings ? Regards. Benoît -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d971709f-abdf-4845-8662-01183c5883ec%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d971709f-abdf-4845-8662-01183c5883ec%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Binh\_Ly](https://avatars.discourse-cdn.com/v4/letter/b/ce7236/32.png) [@Binh\_Ly](https://discuss.elastic.co/u/Binh_Ly)\ **Post date:** [February 25, 2014, 3:25pm UTC](https://discuss.elastic.co/t/nodes-spend-all-time-in-ramusageestimator-after-upgrade-to-0-90-11/16012/2 "2014-02-25T15:25:59Z")
This is a known issue and will be fixed shortly. For now, what you can do is run \_optimize on all your indexes and set max\_num\_segments to 1, like below. Note that this may take a while depending on the size of your indexes. [http://localhost:9200/\_optimize?max\_num\_segments](http://localhost:9200/_optimize?max_num_segments) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6d794291-eca6-46cb-93e8-d45a513990d3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6d794291-eca6-46cb-93e8-d45a513990d3%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Benoit](https://avatars.discourse-cdn.com/v4/letter/b/5daacb/32.png) [@Benoit](https://discuss.elastic.co/u/Benoit)\ **Post date:** [February 25, 2014, 3:38pm UTC](https://discuss.elastic.co/t/nodes-spend-all-time-in-ramusageestimator-after-upgrade-to-0-90-11/16012/3 "2014-02-25T15:38:35Z")
I forgot to say that one consequence is that the 'head' plugin interface remain empty. The following request timeout : - \_status - stats?all=true - \_nodes How to have some information on the cluster in this conditions ? Benoît -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c1511903-f9c8-4332-8451-ed10aaa0fcad%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c1511903-f9c8-4332-8451-ed10aaa0fcad%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Benoit](https://avatars.discourse-cdn.com/v4/letter/b/5daacb/32.png) [@Benoit](https://discuss.elastic.co/u/Benoit)\ **Post date:** [February 25, 2014, 4:50pm UTC](https://discuss.elastic.co/t/nodes-spend-all-time-in-ramusageestimator-after-upgrade-to-0-90-11/16012/4 "2014-02-25T16:50:47Z")
Thank you Binh Ly, On Tuesday, February 25, 2014 4:25:59 PM UTC+1, Binh Ly wrote: > This is a known issue and will be fixed shortly. For now, what you can do > is run \_optimize on all your indexes and set max\_num\_segments to 1, like > below. Note that this may take a while depending on the size of your > indexes. > > [http://localhost:9200/\_optimize?max\_num\_segments](http://localhost:9200/_optimize?max_num_segments) > > Your suggestion confirm what Jörg Prante said here > [https://groups.google.com/d/msg/elasticsearch/7mrDhqe6LEo/3gjOJka85OYJ](https://groups.google.com/d/msg/elasticsearch/7mrDhqe6LEo/3gjOJka85OYJ) > This is a problem with Lucene segment of version 3.x I have around 1T of index, so I'm not really happy to run optimize, I will try on one of the smallest index. If I stop all the request to the statistics API, I should see the load decreasing ? Regards. Benoît -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/99b02139-3d02-434d-a3e4-724b876c3a27%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/99b02139-3d02-434d-a3e4-724b876c3a27%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Benoit](https://avatars.discourse-cdn.com/v4/letter/b/5daacb/32.png) [@Benoit](https://discuss.elastic.co/u/Benoit)\ **Post date:** [February 26, 2014, 1:34pm UTC](https://discuss.elastic.co/t/nodes-spend-all-time-in-ramusageestimator-after-upgrade-to-0-90-11/16012/5 "2014-02-26T13:34:34Z")
The release of last version 0.90.12 solve (or at least hide) this problem. Thanks to the team ! Benoît On Tuesday, February 25, 2014 5:50:47 PM UTC+1, Benoît wrote: > Thank you Binh Ly, > > On Tuesday, February 25, 2014 4:25:59 PM UTC+1, Binh Ly wrote: > > > This is a known issue and will be fixed shortly. For now, what you can do > > is run \_optimize on all your indexes and set max\_num\_segments to 1, like > > below. Note that this may take a while depending on the size of your > > indexes. > > > > [http://localhost:9200/\_optimize?max\_num\_segments](http://localhost:9200/_optimize?max_num_segments) > > > > Your suggestion confirm what Jörg Prante said here > > [https://groups.google.com/d/msg/elasticsearch/7mrDhqe6LEo/3gjOJka85OYJ](https://groups.google.com/d/msg/elasticsearch/7mrDhqe6LEo/3gjOJka85OYJ) > > This is a problem with Lucene segment of version 3.x > > I have around 1T of index, so I'm not really happy to run optimize, I will > try on one of the smallest index. > > If I stop all the request to the statistics API, I should see the load > decreasing ? > > Regards. > > Benoît -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/caf26d0b-73b0-436d-8253-b0298ec0b285%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/caf26d0b-73b0-436d-8253-b0298ec0b285%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:47am UTC](https://discuss.elastic.co/t/nodes-spend-all-time-in-ramusageestimator-after-upgrade-to-0-90-11/16012/6 "2017-07-06T01:47:14Z")