I have 3 nodes 2 masters with data and 1 is master=false and data=false used for facets,aggregation.
All nodes are having 30gb ram with 16gb allocated to elastic search and remaining OS
I have some 600 indices with 5 shards each storing content of documents with vector=true.
Now when I execute a simple more like this query,first 2 nodes starts eating up memory resulting into out of memory exception.Am i missing any settings? or is this a bug?
This is the error log
[2015-06-12 04:19:35,729][DEBUG][action.search.type ] [Master1] [113] Failed to execute fetch phase
org.elasticsearch.ElasticsearchException: Java heap space
at org.elasticsearch.ExceptionsHelper.convertToRuntime(ExceptionsHelper.java:44)
at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:513)
at org.elasticsearch.search.action.SearchServiceTransportAction$17.call(SearchServiceTransportAction.java:452)
at org.elasticsearch.search.action.SearchServiceTransportAction$17.call(SearchServiceTransportAction.java:449)
at org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:559)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOfRange(Unknown Source)
at java.lang.String.(Unknown Source)
at java.lang.StringBuilder.toString(Unknown Source)
at org.elasticsearch.common.jackson.core.util.TextBuffer.contentsAsString(TextBuffer.java:349)
at org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:281)
at org.elasticsearch.common.xcontent.json.JsonXContentParser.text(JsonXContentParser.java:86)
at org.elasticsearch.common.xcontent.support.AbstractXContentParser.readValue(AbstractXContentParser.java:293)
at org.elasticsearch.common.xcontent.support.AbstractXContentParser.readMap(AbstractXContentParser.java:275)
at org.elasticsearch.common.xcontent.support.AbstractXContentParser.readMap(AbstractXContentParser.java:254)
at org.elasticsearch.common.xcontent.support.AbstractXContentParser.map(AbstractXContentParser.java:208)
at org.elasticsearch.common.xcontent.support.AbstractXContentParser.mapAndClose(AbstractXContentParser.java:219)
at org.elasticsearch.common.xcontent.XContentHelper.convertToMap(XContentHelper.java:125)
at org.elasticsearch.common.xcontent.XContentHelper.convertToMap(XContentHelper.java:79)
at org.elasticsearch.search.lookup.SourceLookup.sourceAsMapAndType(SourceLookup.java:87)
at org.elasticsearch.search.lookup.SourceLookup.loadSourceIfNeeded(SourceLookup.java:63)
at org.elasticsearch.search.lookup.SourceLookup.extractRawValues(SourceLookup.java:145)
at org.elasticsearch.search.fetch.FetchPhase.createSearchHit(FetchPhase.java:245)
at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:182)
at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:501)