# Elasticsearch hot threads Interpretation

**URL:** <https://discuss.elastic.co/t/elasticsearch-hot-threads-interpretation/79115>\
**Category:** Elasticsearch\
**Created:** [March 18, 2017, 5:20am UTC](https://discuss.elastic.co/t/elasticsearch-hot-threads-interpretation/79115 "2017-03-18T05:20:46Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vinayak\_K.S](https://avatars.discourse-cdn.com/v4/letter/v/b5ac83/32.png) [@Vinayak\_K.S](https://discuss.elastic.co/u/Vinayak_K.S)\
**Post date:** [March 18, 2017, 5:20am UTC](https://discuss.elastic.co/t/elasticsearch-hot-threads-interpretation/79115/1 "2017-03-18T05:20:46Z")

</div>

Hi,  
I I am having ES(version 1.7.5) cluster of 6 nodes, 3 is kept for read and rest for write. I have a scenario where the usage of parent child concept with bulk indexing is [required.At](http://required.At) irregular intervals CPU usage of majority of nodes go above 90%. I couldn't find any direct relation to any of my search or indexing activity. When i ran hot threads API i get following results.

101.4% (506.7ms out of 500ms) cpu usage by thread 'elasticsearch[node1][search][T#4]'  
2/10 snapshots sharing following 47 elements  
org.apache.lucene.util.packed.Packed64.set(Packed64.java:223)  
org.apache.lucene.util.packed.PackedLongValues$Builder.pack(PackedLongValues.java:268)  
org.apache.lucene.util.packed.DeltaPackedLongValues$Builder.pack(DeltaPackedLongValues.java:89)  
org.apache.lucene.util.packed.MonotonicLongValues$Builder.pack(MonotonicLongValues.java:91)  
org.apache.lucene.util.packed.PackedLongValues$Builder.pack(PackedLongValues.java:244)  
org.apache.lucene.util.packed.PackedLongValues$Builder.add(PackedLongValues.java:227)  
org.apache.lucene.index.MultiDocValues$OrdinalMap.(MultiDocValues.java:550)  
org.apache.lucene.index.MultiDocValues$OrdinalMap.build(MultiDocValues.java:482)  
org.apache.lucene.index.MultiDocValues$OrdinalMap.build(MultiDocValues.java:446)  
org.elasticsearch.index.fielddata.plain.ParentChildIndexFieldData.buildOrdinalMap(ParentChildIndexFieldData.java:282)  
org.elasticsearch.index.fielddata.plain.ParentChildIndexFieldData.localGlobalDirect(ParentChildIndexFieldData.java:312)  
org.elasticsearch.index.fielddata.plain.ParentChildIndexFieldData.localGlobalDirect(ParentChildIndexFieldData.java:65)  
org.elasticsearch.indices.fielddata.cache.IndicesFieldDataCache$IndexFieldCache$2.call(IndicesFieldDataCache.java:211)  
org.elasticsearch.indices.fielddata.cache.IndicesFieldDataCache$IndexFieldCache$2.call(IndicesFieldDataCache.java:199)  
org.elasticsearch.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4742)  
org.elasticsearch.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)  
org.elasticsearch.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)  
org.elasticsearch.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)  
org.elasticsearch.common.cache.LocalCache$Segment.get(LocalCache.java:2197)  
org.elasticsearch.common.cache.LocalCache.get(LocalCache.java:3937)  
org.elasticsearch.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4739)  
org.elasticsearch.indices.fielddata.cache.IndicesFieldDataCache$IndexFieldCache.load(IndicesFieldDataCache.java:199)  
org.elasticsearch.index.fielddata.plain.ParentChildIndexFieldData.loadGlobal(ParentChildIndexFieldData.java:267)  
org.elasticsearch.index.search.child.ChildrenConstantScoreQuery.createWeight(ChildrenConstantScoreQuery.java:96)  
org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:684)  
org.elasticsearch.search.internal.ContextIndexSearcher.createNormalizedWeight(ContextIndexSearcher.java:133)  
org.elasticsearch.index.search.child.CustomQueryWrappingFilter.getDocIdSet(CustomQueryWrappingFilter.java:72)  
org.elasticsearch.common.lucene.search.ApplyAcceptedDocsFilter.getDocIdSet(ApplyAcceptedDocsFilter.java:46)  
org.apache.lucene.search.ConstantScoreQuery$ConstantWeight.scorer(ConstantScoreQuery.java:157)  
org.apache.lucene.search.Weight.bulkScorer(Weight.java:131)  
org.apache.lucene.search.ConstantScoreQuery$ConstantWeight.bulkScorer(ConstantScoreQuery.java:141)  
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:618)  
org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:191)  
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491)  
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)  
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)  
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)  
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:157)  
org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:301)  
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:312)  
org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:776)  
org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:767)  
org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.doRun(MessageChannelHandler.java:279)  
org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:36)  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
java.lang.Thread.run(Thread.java:745)

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [March 20, 2017, 9:13am UTC](https://discuss.elastic.co/t/elasticsearch-hot-threads-interpretation/79115/2 "2017-03-20T09:13:26Z")

</div>

Hey,

this looks, as if during a search the data structure for parent/child is being recreated. You can change this to ensure it is loaded eagerly using global ordinals for example (note, be careful with this option if you have high indexing rates, take your time to play around with it).

See [https://www.elastic.co/guide/en/elasticsearch/reference/1.7/fielddata-formats.html#\_global\_ordinals](https://www.elastic.co/guide/en/elasticsearch/reference/1.7/fielddata-formats.html#_global_ordinals)

--Alex

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 17, 2017, 9:13am UTC](https://discuss.elastic.co/t/elasticsearch-hot-threads-interpretation/79115/3 "2017-04-17T09:13:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
