# OutOfMemory error **URL:** \ **Category:** Elasticsearch\ **Created:** [January 2, 2019, 8:22am UTC](https://discuss.elastic.co/t/outofmemory-error/162608 "2019-01-02T08:22:09Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![pranav24](https://avatars.discourse-cdn.com/v4/letter/p/b19c9b/32.png) [@pranav24](https://discuss.elastic.co/u/pranav24)\ **Post date:** [January 2, 2019, 8:22am UTC](https://discuss.elastic.co/t/outofmemory-error/162608/1 "2019-01-02T08:22:09Z")
Hi, I'm facing the issue that my Elasticsearch data nodes shutdown automatically for some searches. Elasticsearch version - 6.3.2 Ram - 8gb Heap size - 1gb Documents - 7000(max) This is the error -\> ``` at java.lang.OutOfMemoryError.(OutOfMemoryError.java:48) at java.util.TreeMap.put(TreeMap.java:577) Local Variable: java.util.TreeMap#2 Local Variable: java.lang.Integer#160 Local Variable: java.util.TreeMap$Entry#149 Local Variable: java.lang.Integer#294 at org.apache.lucene.util.automaton.SortedIntSet.incr(SortedIntSet.java:53) at org.apache.lucene.util.automaton.Operations.determinize(Operations.java:780) Local Variable: org.apache.lucene.util.automaton.Automaton$Builder#2 Local Variable: java.util.HashMap#3 Local Variable: java.util.ArrayDeque#2 Local Variable: org.apache.lucene.util.automaton.SortedIntSet#2 Local Variable: org.apache.lucene.util.automaton.Operations$PointTransitionSet#2 Local Variable: int[]#8065 Local Variable: org.apache.lucene.util.automaton.Automaton#1980 at org.apache.lucene.util.automaton.Operations.getCommonSuffixBytesRef(Operations.java:1155) at org.apache.lucene.util.automaton.CompiledAutomaton.(CompiledAutomaton.java:238) Local Variable: org.apache.lucene.util.automaton.CompiledAutomaton#1108 Local Variable: org.apache.lucene.util.automaton.Automaton#1979 at org.apache.lucene.search.AutomatonQuery.(AutomatonQuery.java:104) at org.apache.lucene.search.AutomatonQuery.(AutomatonQuery.java:81) at org.apache.lucene.search.WildcardQuery.(WildcardQuery.java:67) at org.apache.lucene.queryparser.classic.QueryParserBase.newWildcardQuery(QueryParserBase.java:644) at org.apache.lucene.queryparser.classic.QueryParserBase.getWildcardQuery(QueryParserBase.java:703) at org.elasticsearch.index.search.QueryStringQueryParser.getWildcardQuerySingle(QueryStringQueryParser.java:666) Local Variable: java.lang.String#170193 at org.elasticsearch.index.search.QueryStringQueryParser.getWildcardQuery(QueryStringQueryParser.java:640) Local Variable: java.util.Collections$UnmodifiableMap$UnmodifiableEntrySet$1#2 Local Variable: java.lang.String#220606 Local Variable: java.util.ArrayList#9417 Local Variable: java.util.Collections$UnmodifiableMap$UnmodifiableEntrySet$UnmodifiableEntry#4 at org.apache.lucene.queryparser.classic.QueryParserBase.handleBareTokenQuery(QueryParserBase.java:820) at org.apache.lucene.queryparser.classic.QueryParser.Term(QueryParser.java:469) at org.apache.lucene.queryparser.classic.QueryParser.Clause(QueryParser.java:355) at org.apache.lucene.queryparser.classic.QueryParser.Query(QueryParser.java:244) Local Variable: java.util.ArrayList#9418 at org.apache.lucene.queryparser.classic.QueryParser.TopLevelQuery(QueryParser.java:215) at org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:109) Local Variable: java.lang.String#220898 Local Variable: org.elasticsearch.index.search.QueryStringQueryParser#3 at org.elasticsearch.index.search.QueryStringQueryParser.parse(QueryStringQueryParser.java:786) at org.elasticsearch.index.query.QueryStringQueryBuilder.doToQuery(QueryStringQueryBuilder.java:1041) at org.elasticsearch.index.query.AbstractQueryBuilder.toQuery(AbstractQueryBuilder.java:98) Local Variable: org.elasticsearch.index.query.QueryStringQueryBuilder#70 at org.elasticsearch.index.query.BoolQueryBuilder.addBooleanClauses(BoolQueryBuilder.java:404) Local Variable: java.util.ArrayList$Itr#3 at org.elasticsearch.index.query.BoolQueryBuilder.doToQuery(BoolQueryBuilder.java:378) Local Variable: org.apache.lucene.search.BooleanQuery$Builder#3 at org.elasticsearch.index.query.AbstractQueryBuilder.toQuery(AbstractQueryBuilder.java:98) Local Variable: org.elasticsearch.index.query.BoolQueryBuilder#54 at org.elasticsearch.index.query.BoolQueryBuilder.addBooleanClauses(BoolQueryBuilder.java:404) Local Variable: java.util.ArrayList$Itr#4 at org.elasticsearch.index.query.BoolQueryBuilder.doToQuery(BoolQueryBuilder.java:380) Local Variable: org.apache.lucene.search.BooleanQuery$Builder#4 at org.elasticsearch.index.query.AbstractQueryBuilder.toQuery(AbstractQueryBuilder.java:98) at org.elasticsearch.index.query.QueryShardContext.lambda$toQuery$2(QueryShardContext.java:304) at org.elasticsearch.index.query.QueryShardContext$$Lambda$2900.apply() at org.elasticsearch.index.query.QueryShardContext.toQuery(QueryShardContext.java:316) Local Variable: org.elasticsearch.index.query.QueryShardContext$$Lambda$2900#2 at org.elasticsearch.index.query.QueryShardContext.toQuery(QueryShardContext.java:303) Local Variable: org.elasticsearch.index.query.BoolQueryBuilder#53 at org.elasticsearch.search.SearchService.parseSource(SearchService.java:724) Local Variable: java.util.HashMap#63550 Local Variable: org.elasticsearch.search.builder.SearchSourceBuilder#14 Local Variable: org.elasticsearch.index.query.QueryShardContext#3 at org.elasticsearch.search.SearchService.createContext(SearchService.java:575) Local Variable: org.elasticsearch.search.DefaultSearchContext#2 at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:551) at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:347) Local Variable: org.elasticsearch.action.search.SearchTask#11 at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:333) at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:329) Local Variable: org.elasticsearch.search.internal.ShardSearchTransportRequest#14 Local Variable: org.elasticsearch.search.SearchService$2#11 at org.elasticsearch.search.SearchService$3.doRun(SearchService.java:1019) Local Variable: org.elasticsearch.search.SearchService$3#11 ``` On evaluating the ".hprof" file for heap dump these are the problem suspects: ![Screenshot%20(40)](https://us1.discourse-cdn.com/elastic/original/3X/6/9/692e53bb304d9fc9781ac5a033b1a5ead7c2da8e.png) Can you guys tell me what is the problem here? Is it because of my queries or less heap size? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 2, 2019, 11:38am UTC](https://discuss.elastic.co/t/outofmemory-error/162608/2 "2019-01-02T11:38:55Z")
I'd first use up to half of the RAM available as the heap size. 4 gb here. ---
**Author:** ![pranav24](https://avatars.discourse-cdn.com/v4/letter/p/b19c9b/32.png) [@pranav24](https://discuss.elastic.co/u/pranav24)\ **Post date:** [January 2, 2019, 11:43am UTC](https://discuss.elastic.co/t/outofmemory-error/162608/3 "2019-01-02T11:43:51Z")
Thanks @dadoonet I understand Heap size could be one of the reasons, Is there anything other than this that can cause out of memory issues? Will having fields over 1000 in each index affect memory? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 2, 2019, 2:51pm UTC](https://discuss.elastic.co/t/outofmemory-error/162608/4 "2019-01-02T14:51:03Z")
Yes. ---
**Author:** ![pranav24](https://avatars.discourse-cdn.com/v4/letter/p/b19c9b/32.png) [@pranav24](https://discuss.elastic.co/u/pranav24)\ **Post date:** [January 3, 2019, 6:06am UTC](https://discuss.elastic.co/t/outofmemory-error/162608/5 "2019-01-03T06:06:27Z")
> [@pranav24](#): > > Is there anything other than this that can cause out of memory issues? what can they be? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 3, 2019, 6:26am UTC](https://discuss.elastic.co/t/outofmemory-error/162608/6 "2019-01-03T06:26:01Z")
As you have a small number of documents, that's probably more related to the content, fields, query you are running.... ---
**Author:** ![pranav24](https://avatars.discourse-cdn.com/v4/letter/p/b19c9b/32.png) [@pranav24](https://discuss.elastic.co/u/pranav24)\ **Post date:** [January 3, 2019, 6:39am UTC](https://discuss.elastic.co/t/outofmemory-error/162608/7 "2019-01-03T06:39:54Z")
Thank you @dadoonet **Question 1** : As I could see in a stacktrace, It is showing wildcard query. > ``` > at org.apache.lucene.search.AutomatonQuery.(AutomatonQuery.java:104) > at org.apache.lucene.search.AutomatonQuery.(AutomatonQuery.java:81) > at org.apache.lucene.search.WildcardQuery.(WildcardQuery.java:67) > at org.apache.lucene.queryparser.classic.QueryParserBase.newWildcardQuery(QueryParserBase.java:644) > at org.apache.lucene.queryparser.classic.QueryParserBase.getWildcardQuery(QueryParserBase.java:703) > at org.elasticsearch.index.search.QueryStringQueryParser.getWildcardQuerySingle(QueryStringQueryParser.java:666) > > ``` As a wildcard query internally does a lot of computation, So could it be the reason for Out of memory? **Question 2** : The problem suspects which are shown by a heap dump analysis: > ``` > One instance of "org.elasticsearch.search.SearchService" loaded by "sun.misc.Launcher$AppClassLoader @ Oxca660000" occupies 484,600,608 (46.12%) bytes. The memory is accumulated in one instance of "java.util.concurrent.ConcurrentHashMap$Node[]" loaded by "". > Keywords java.util.concurrent.ConcurrentHashMap$Nodep sun.misc.Launcher$AppClassLoader @ Oxca660000 org.elasticsearch.search.SearchService > > ``` ConcurrentHashMap object is occupying my lot of memory. Can you tell when this object is created and used? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 3, 2019, 7:45am UTC](https://discuss.elastic.co/t/outofmemory-error/162608/8 "2019-01-03T07:45:50Z")
> [@pranav24](#): > > So could it be the reason for Out of memory? If you have a lot of terms I guess so. What is your query ? > [@pranav24](#): > > Can you tell when this object is created and used? I can't. But have a look at [Elastic Mapping explosion - #7 by Mark\_Harwood](https://discuss.elastic.co/t/elastic-mapping-explosion/162647/7) ---
**Author:** ![pranav24](https://avatars.discourse-cdn.com/v4/letter/p/b19c9b/32.png) [@pranav24](https://discuss.elastic.co/u/pranav24)\ **Post date:** [January 3, 2019, 8:16am UTC](https://discuss.elastic.co/t/outofmemory-error/162608/9 "2019-01-03T08:16:08Z")
Thanks for the reply As I see in my code, I have a query running with wildcard ``` { "from" : 0, "size" : 5,"_source": {"includes": [] }, "query": { "bool": {"should": [ { "bool": {"must": [ { "query_string": { "query":"*only* *search* *for* *pranav*", "fields": [] }}]}}],"minimum_should_match":1}},"highlight": { "type" : "unified", "fragment_size" : 0, "order":"score", "number_of_fragments" :4, "fields": { "*": { } } }} ``` This query continuously hits my Elastic server when a new character is typed. Something like AutoSuggestion but not exactly the same. Can this query be the reason fro my OutOfMemory issue? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 3, 2019, 8:37am UTC](https://discuss.elastic.co/t/outofmemory-error/162608/10 "2019-01-03T08:37:10Z")
Probably. You are using wildcard queries which not recommended. Anyway, as we saw you have plenty of RAM available, just upgrade the HEAP to 4 gb and you should be good. ---
**Author:** ![Bernt\_Rostad](https://avatars.discourse-cdn.com/v4/letter/b/3ab097/32.png) [@Bernt\_Rostad](https://discuss.elastic.co/u/Bernt_Rostad)\ **Post date:** [January 3, 2019, 9:51am UTC](https://discuss.elastic.co/t/outofmemory-error/162608/11 "2019-01-03T09:51:57Z")
> [@pranav24](#): > > Can this query be the reason fro my OutOfMemory issue? I've seen a single query, with an asterisk prefix, eat all 29 G heap size on a node, causing it to crash with OutOfMemory, so yes - wildcards can cause OutOfMemory errors. Even if you add more Java Heap Size. I have tried to play with the [circuit breaker](https://www.elastic.co/guide/en/elasticsearch/reference/current/circuit-breaker.html) settings, to terminate such dangerous queries, but without luck so far. So your best line of action is probably to prevent such queries from being run in the first place but also have a good system for detecting node crashes and restarting them to avoid a red cluster state. ---
**Author:** ![pranav24](https://avatars.discourse-cdn.com/v4/letter/p/b19c9b/32.png) [@pranav24](https://discuss.elastic.co/u/pranav24)\ **Post date:** [January 3, 2019, 10:25am UTC](https://discuss.elastic.co/t/outofmemory-error/162608/12 "2019-01-03T10:25:17Z")
Thank you so much @dadoonet @Bernt_Rostad for your response. I have increased the HEAP size and didn't get any issue from past 2 days. I am also planning to reduce the use cases of a wild card. Just for curiosity, I want to know something about Mapping Explosion. As I have about 1800 fields, I know this number is too large. As Elasticsearch provides functionality to increase the number of fields. So by increasing it to a large value can cause any issues? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 31, 2019, 10:25am UTC](https://discuss.elastic.co/t/outofmemory-error/162608/13 "2019-01-31T10:25:42Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.