# Heap OOM.. after running for a long time
**URL:** https://discuss.elastic.co/t/heap-oom-after-running-for-a-long-time/14699
**Category:** Elasticsearch
**Created:** [December 4, 2013, 10:20am UTC](https://discuss.elastic.co/t/heap-oom-after-running-for-a-long-time/14699 "2013-12-04T10:20:13Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@Klavs\_Klavsen](https://discuss.elastic.co/u/Klavs_Klavsen)
#### Post date: [December 4, 2013, 10:20am UTC](https://discuss.elastic.co/t/heap-oom-after-running-for-a-long-time/14699/1 "2013-12-04T10:20:13Z")
elasticsearch 0.90.6 just crashed on me.. ☹
I have given it 16GB Heap space.. -Xms16051M -Xmx16051M -Xss256k
Error in log was:
[2013-12-04 11:05:20,111][DEBUG][action.search.type] [Necrodamus]
[logstash-2013.11.20][2], node[vSEeksIDSdiqtqDFgwQsNQ], [P], s[STARTED]:
Failed to execute [org.elasticsearch.action.search.SearchRequest@5f1223b5]
org.elasticsearch.search.query.QueryPhaseExecutionException:
[logstash-2013.11.20][2]: query[ConstantScore(_:_)],from[0],size[0]: Query
Failed [Failed to execute main query]
at
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:123)
at
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:233)
at
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)
at
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.elasticsearch.ElasticSearchException:
java.lang.OutOfMemoryError: Java heap space
at
org.elasticsearch.index.fielddata.AbstractIndexFieldData.load(AbstractIndexFieldData.java:60)
at
org.elasticsearch.index.fielddata.plain.PackedArrayIndexFieldData.load(PackedArrayIndexFieldData.java:49)
at
org.elasticsearch.search.facet.datehistogram.CountDateHistogramFacetExecutor$Collector.setNextReader(CountDateHistogramFacetExecutor.java:88)
at
org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:68)
at
org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:618)
at
org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:167)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
at
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:119)
... 9 more
Caused by: org.elasticsearch.common.util.concurrent.ExecutionError:
java.lang.OutOfMemoryError: Java heap space
at
org.elasticsearch.common.cache.LocalCache$Segment.get(LocalCache.java:2197)
at
org.elasticsearch.common.cache.LocalCache.get(LocalCache.java:3932)
at
org.elasticsearch.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4721)
at
org.elasticsearch.indices.fielddata.cache.IndicesFieldDataCache$IndexFieldCache.load(IndicesFieldDataCache.java:143)
at
org.elasticsearch.index.fielddata.AbstractIndexFieldData.load(AbstractIndexFieldData.java:53)
... 20 more
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.apache.lucene.util.packed.Packed64.(Packed64.java:73)
at
org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:1084)
at
org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:1051)
at
org.elasticsearch.index.fielddata.plain.PackedArrayIndexFieldData.loadDirect(PackedArrayIndexFieldData.java:162)
at
org.elasticsearch.index.fielddata.plain.PackedArrayIndexFieldData.loadDirect(PackedArrayIndexFieldData.java:49)
at
org.elasticsearch.indices.fielddata.cache.IndicesFieldDataCache$IndexFieldCache$1.call(IndicesFieldDataCache.java:149)
at
org.elasticsearch.indices.fielddata.cache.IndicesFieldDataCache$IndexFieldCache$1.call(IndicesFieldDataCache.java:143)
at
org.elasticsearch.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4724)
at
org.elasticsearch.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3522)
at
org.elasticsearch.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2315)
at
org.elasticsearch.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2278)
at
org.elasticsearch.common.cache.LocalCache$Segment.get(LocalCache.java:2193)
at
org.elasticsearch.common.cache.LocalCache.get(LocalCache.java:3932)
at
org.elasticsearch.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4721)
at
org.elasticsearch.indices.fielddata.cache.IndicesFieldDataCache$IndexFieldCache.load(IndicesFieldDataCache.java:143)
at
org.elasticsearch.index.fielddata.AbstractIndexFieldData.load(AbstractIndexFieldData.java:53)
at
org.elasticsearch.index.fielddata.plain.PackedArrayIndexFieldData.load(PackedArrayIndexFieldData.java:49)
at
org.elasticsearch.search.facet.datehistogram.CountDateHistogramFacetExecutor$Collector.setNextReader(CountDateHistogramFacetExecutor.java:88)
at
org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:68)
at
org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:618)
at
org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:167)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
at
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:119)
at
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:233)
at
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)
at
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)
Any hints as to what I could do to improve my settings are welcome. My
current elasticsearch.yml looks like this:
index.warmer.enabled: false
action.disable\_delete\_all\_indices: true
indices.memory.index\_buffer\_size: 50%
threadpool.search.type: fixed
threadpool.search.size: 20
threadpool.search.queue\_size: 100
threadpool.index.type: fixed
threadpool.index.size: 60
threadpool.index.queue\_size: 200
bootstrap.mlockall: true
and I use ES for logstash only.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/7f0a3dc4-6af6-490d-af98-8e45c273ae67%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7f0a3dc4-6af6-490d-af98-8e45c273ae67%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@spinscale](https://discuss.elastic.co/u/spinscale)
#### Post date: [December 4, 2013, 10:43am UTC](https://discuss.elastic.co/t/heap-oom-after-running-for-a-long-time/14699/2 "2013-12-04T10:43:09Z")
Hey,
looks the the fielddata outgrew your heap. See more about fielddata at
> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.
This data structure grows with the amount of data your are querying. You
can monitor it (and watching it grow over time) using the nodes stats, see
> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.
Make sure that you are not facetting on analyzed fields (check your mapping
and your queries for that), as this is a common cause for going OOM.
In your logstash case you could make sure, that you do not store fields as
analyzed by using an index template.
--Alex
On Wed, Dec 4, 2013 at 11:20 AM, Klavs Klavsen [klavs@enableit.dk](mailto:klavs@enableit.dk) wrote:
> elasticsearch 0.90.6 just crashed on me.. ☹
>
> I have given it 16GB Heap space.. -Xms16051M -Xmx16051M -Xss256k
>
> Error in log was:
> [2013-12-04 11:05:20,111][DEBUG][action.search.type] [Necrodamus]
> [logstash-2013.11.20][2], node[vSEeksIDSdiqtqDFgwQsNQ], [P], s[STARTED]:
> Failed to execute [org.elasticsearch.action.search.SearchRequest@5f1223b5]
> org.elasticsearch.search.query.QueryPhaseExecutionException:
> [logstash-2013.11.20][2]: query[ConstantScore(_:_)],from[0],size[0]: Query
> Failed [Failed to execute main query]
> at
> org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:123)
> at
> org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:233)
> at
> org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)
> at
> org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
> at
> org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)
> at
> org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)
> at
> org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
>
> Caused by: org.elasticsearch.ElasticSearchException:
> java.lang.OutOfMemoryError: Java heap space
> at
> org.elasticsearch.index.fielddata.AbstractIndexFieldData.load(AbstractIndexFieldData.java:60)
> at
> org.elasticsearch.index.fielddata.plain.PackedArrayIndexFieldData.load(PackedArrayIndexFieldData.java:49)
> at
> org.elasticsearch.search.facet.datehistogram.CountDateHistogramFacetExecutor$Collector.setNextReader(CountDateHistogramFacetExecutor.java:88)
> at
> org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:68)
> at
> org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:618)
> at
> org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:167)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
> at
> org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:119)
> ... 9 more
> Caused by: org.elasticsearch.common.util.concurrent.ExecutionError:
> java.lang.OutOfMemoryError: Java heap space
> at
> org.elasticsearch.common.cache.LocalCache$Segment.get(LocalCache.java:2197)
> at
> org.elasticsearch.common.cache.LocalCache.get(LocalCache.java:3932)
> at
> org.elasticsearch.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4721)
> at
> org.elasticsearch.indices.fielddata.cache.IndicesFieldDataCache$IndexFieldCache.load(IndicesFieldDataCache.java:143)
> at
> org.elasticsearch.index.fielddata.AbstractIndexFieldData.load(AbstractIndexFieldData.java:53)
> ... 20 more
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at org.apache.lucene.util.packed.Packed64.(Packed64.java:73)
> at
> org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:1084)
> at
> org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:1051)
> at
> org.elasticsearch.index.fielddata.plain.PackedArrayIndexFieldData.loadDirect(PackedArrayIndexFieldData.java:162)
> at
> org.elasticsearch.index.fielddata.plain.PackedArrayIndexFieldData.loadDirect(PackedArrayIndexFieldData.java:49)
> at
> org.elasticsearch.indices.fielddata.cache.IndicesFieldDataCache$IndexFieldCache$1.call(IndicesFieldDataCache.java:149)
> at
> org.elasticsearch.indices.fielddata.cache.IndicesFieldDataCache$IndexFieldCache$1.call(IndicesFieldDataCache.java:143)
> at
> org.elasticsearch.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4724)
> at
> org.elasticsearch.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3522)
> at
> org.elasticsearch.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2315)
> at
> org.elasticsearch.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2278)
> at
> org.elasticsearch.common.cache.LocalCache$Segment.get(LocalCache.java:2193)
> at
> org.elasticsearch.common.cache.LocalCache.get(LocalCache.java:3932)
> at
> org.elasticsearch.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4721)
> at
> org.elasticsearch.indices.fielddata.cache.IndicesFieldDataCache$IndexFieldCache.load(IndicesFieldDataCache.java:143)
> at
> org.elasticsearch.index.fielddata.AbstractIndexFieldData.load(AbstractIndexFieldData.java:53)
> at
> org.elasticsearch.index.fielddata.plain.PackedArrayIndexFieldData.load(PackedArrayIndexFieldData.java:49)
> at
> org.elasticsearch.search.facet.datehistogram.CountDateHistogramFacetExecutor$Collector.setNextReader(CountDateHistogramFacetExecutor.java:88)
> at
> org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:68)
> at
> org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:618)
> at
> org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:167)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
> at
> org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:119)
> at
> org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:233)
> at
> org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)
> at
> org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
> at
> org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)
> at
> org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)
>
> Any hints as to what I could do to improve my settings are welcome. My
> current elasticsearch.yml looks like this:
> index.warmer.enabled: false
> action.disable\_delete\_all\_indices: true
> indices.memory.index\_buffer\_size: 50%
> threadpool.search.type: fixed
> threadpool.search.size: 20
> threadpool.search.queue\_size: 100
>
> threadpool.index.type: fixed
> threadpool.index.size: 60
> threadpool.index.queue\_size: 200
> bootstrap.mlockall: true
>
> and I use ES for logstash only.
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> To view this discussion on the web visit
> [https://groups.google.com/d/msgid/elasticsearch/7f0a3dc4-6af6-490d-af98-8e45c273ae67%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7f0a3dc4-6af6-490d-af98-8e45c273ae67%40googlegroups.com)
> .
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGCwEM\_ZAWPEq6wfhKWnq41UTD18PpNV9xRQebrLOooz7OS8Hg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGCwEM_ZAWPEq6wfhKWnq41UTD18PpNV9xRQebrLOooz7OS8Hg%40mail.gmail.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Klavs\_Klavsen](https://discuss.elastic.co/u/Klavs_Klavsen)
#### Post date: [December 4, 2013, 11:23am UTC](https://discuss.elastic.co/t/heap-oom-after-running-for-a-long-time/14699/3 "2013-12-04T11:23:44Z")
Hi Alexander,
thank you very much for the tips.
As I understand it, I should only analyze fields, for which I would like to
search on "subsets" of the content.. (other than regex searches.. and
"value comparison" like \> 20.. ). ?
but how complex regex'es can I use? the biggest save on the "syslog"
mesages - would be to not analyze the main "message" field.. and since I
should have pulled out anything of interest into specific fields, that
would probably be ok?
I googled and found an example of what mappings others are using:
[Logstash Elasticsearch Template · GitHub](https://gist.github.com/deverton/2970285)..
All this can minimize my HEAP overhead.. but no matter what, my HEAP
requirements will grow, as the amount of docs grow I gather 🙂
When I add other nodes.. does ES spread the HEAP usage (since it moves
shards around) - so only the local shards, are in the local HEAP ?
So if one node needs 16GB heap.. (and I have replicas = 0).. then adding a
second node would mean they each require ~8GB heap ?
Den onsdag den 4. december 2013 11.43.09 UTC+1 skrev Alexander Reelsen:
> Hey,
>
> looks the the fielddata outgrew your heap. See more about fielddata at
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-fielddata.html)
>
> This data structure grows with the amount of data your are querying. You
> can monitor it (and watching it grow over time) using the nodes stats, see
>
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-nodes-stats.html#field-data)[http://www.google.com/url?q=http%3A%2F%2Fwww.elasticsearch.org%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Fcluster-nodes-stats.html%23field-data&sa=D&sntz=1&usg=AFQjCNEKLZGRa-HFFJ5DPfDVEZSZWOQXSw](http://www.google.com/url?q=http%3A%2F%2Fwww.elasticsearch.org%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Fcluster-nodes-stats.html%23field-data&sa=D&sntz=1&usg=AFQjCNEKLZGRa-HFFJ5DPfDVEZSZWOQXSw)
>
> Make sure that you are not facetting on analyzed fields (check your
> mapping and your queries for that), as this is a common cause for going OOM.
>
> In your logstash case you could make sure, that you do not store fields as
> analyzed by using an index template.
>
> --Alex
>
> On Wed, Dec 4, 2013 at 11:20 AM, Klavs Klavsen \
>
> > wrote:
>
> > elasticsearch 0.90.6 just crashed on me.. ☹
> >
> > I have given it 16GB Heap space.. -Xms16051M -Xmx16051M -Xss256k
> >
> > Error in log was:
> > [2013-12-04 11:05:20,111][DEBUG][action.search.type] [Necrodamus]
> > [logstash-2013.11.20][2], node[vSEeksIDSdiqtqDFgwQsNQ], [P], s[STARTED]:
> > Failed to execute [org.elasticsearch.action.search.SearchRequest@5f1223b5]
> > org.elasticsearch.search.query.QueryPhaseExecutionException:
> > [logstash-2013.11.20][2]: query[ConstantScore(_:_)],from[0],size[0]: Query
> > Failed [Failed to execute main query]
> > at
> > org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:123)
> > at
> > org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:233)
> > at
> > org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)
> > at
> > org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
> > at
> > org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)
> > at
> > org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)
> > at
> > org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)
> > at
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > at
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > at java.lang.Thread.run(Thread.java:744)
> >
> > Caused by: org.elasticsearch.ElasticSearchException:
> > java.lang.OutOfMemoryError: Java heap space
> > at
> > org.elasticsearch.index.fielddata.AbstractIndexFieldData.load(AbstractIndexFieldData.java:60)
> > at
> > org.elasticsearch.index.fielddata.plain.PackedArrayIndexFieldData.load(PackedArrayIndexFieldData.java:49)
> > at
> > org.elasticsearch.search.facet.datehistogram.CountDateHistogramFacetExecutor$Collector.setNextReader(CountDateHistogramFacetExecutor.java:88)
> > at
> > org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:68)
> > at
> > org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)
> > at
> > org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:618)
> > at
> > org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:167)
> > at
> > org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491)
> > at
> > org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
> > at
> > org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
> > at
> > org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
> > at
> > org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:119)
> > ... 9 more
> > Caused by: org.elasticsearch.common.util.concurrent.ExecutionError:
> > java.lang.OutOfMemoryError: Java heap space
> > at
> > org.elasticsearch.common.cache.LocalCache$Segment.get(LocalCache.java:2197)
> > at
> > org.elasticsearch.common.cache.LocalCache.get(LocalCache.java:3932)
> > at
> > org.elasticsearch.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4721)
> > at
> > org.elasticsearch.indices.fielddata.cache.IndicesFieldDataCache$IndexFieldCache.load(IndicesFieldDataCache.java:143)
> > at
> > org.elasticsearch.index.fielddata.AbstractIndexFieldData.load(AbstractIndexFieldData.java:53)
> > ... 20 more
> > Caused by: java.lang.OutOfMemoryError: Java heap space
> > at org.apache.lucene.util.packed.Packed64.(Packed64.java:73)
> > at
> > org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:1084)
> > at
> > org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:1051)
> > at
> > org.elasticsearch.index.fielddata.plain.PackedArrayIndexFieldData.loadDirect(PackedArrayIndexFieldData.java:162)
> > at
> > org.elasticsearch.index.fielddata.plain.PackedArrayIndexFieldData.loadDirect(PackedArrayIndexFieldData.java:49)
> > at
> > org.elasticsearch.indices.fielddata.cache.IndicesFieldDataCache$IndexFieldCache$1.call(IndicesFieldDataCache.java:149)
> > at
> > org.elasticsearch.indices.fielddata.cache.IndicesFieldDataCache$IndexFieldCache$1.call(IndicesFieldDataCache.java:143)
> > at
> > org.elasticsearch.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4724)
> > at
> > org.elasticsearch.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3522)
> > at
> > org.elasticsearch.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2315)
> > at
> > org.elasticsearch.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2278)
> > at
> > org.elasticsearch.common.cache.LocalCache$Segment.get(LocalCache.java:2193)
> > at
> > org.elasticsearch.common.cache.LocalCache.get(LocalCache.java:3932)
> > at
> > org.elasticsearch.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4721)
> > at
> > org.elasticsearch.indices.fielddata.cache.IndicesFieldDataCache$IndexFieldCache.load(IndicesFieldDataCache.java:143)
> > at
> > org.elasticsearch.index.fielddata.AbstractIndexFieldData.load(AbstractIndexFieldData.java:53)
> > at
> > org.elasticsearch.index.fielddata.plain.PackedArrayIndexFieldData.load(PackedArrayIndexFieldData.java:49)
> > at
> > org.elasticsearch.search.facet.datehistogram.CountDateHistogramFacetExecutor$Collector.setNextReader(CountDateHistogramFacetExecutor.java:88)
> > at
> > org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:68)
> > at
> > org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)
> > at
> > org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:618)
> > at
> > org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:167)
> > at
> > org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491)
> > at
> > org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
> > at
> > org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
> > at
> > org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
> > at
> > org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:119)
> > at
> > org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:233)
> > at
> > org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)
> > at
> > org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
> > at
> > org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)
> > at
> > org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)
> >
> > Any hints as to what I could do to improve my settings are welcome. My
> > current elasticsearch.yml looks like this:
> > index.warmer.enabled: false
> > action.disable\_delete\_all\_indices: true
> > indices.memory.index\_buffer\_size: 50%
> > threadpool.search.type: fixed
> > threadpool.search.size: 20
> > threadpool.search.queue\_size: 100
> >
> > threadpool.index.type: fixed
> > threadpool.index.size: 60
> > threadpool.index.queue\_size: 200
> > bootstrap.mlockall: true
> >
> > and I use ES for logstash only.
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > To view this discussion on the web visit
> > [https://groups.google.com/d/msgid/elasticsearch/7f0a3dc4-6af6-490d-af98-8e45c273ae67%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7f0a3dc4-6af6-490d-af98-8e45c273ae67%40googlegroups.com)
> > .
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/7600f344-59a3-400c-9d7d-3c4a0c750149%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7600f344-59a3-400c-9d7d-3c4a0c750149%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 2:03am UTC](https://discuss.elastic.co/t/heap-oom-after-running-for-a-long-time/14699/4 "2017-07-06T02:03:24Z")