Query Failed [Failed to execute main query] OutOfMemoryError: Java heap space

Elastic Stack Elasticsearch
1

Hi i have data in elastic 80 M of data have 52 Go and i search in text with this query

$json_doc = '{
                 "query": {
                    "query_string": {
                        "query": "*' . $sujet . '*",
                        "lenient": true
                    }
                    },
     "size": 10000000,
     "from": 0,
     "fields": [
     "association_key","text"
                    ]
                }';

when I search a word that is in the base is looking less 20K I have my data but as his exceeds 20 K I have this error

[2015-11-14 20:03:04,663][DEBUG][action.search.type       ] [Pixx] [myindex3][2], node[x9gOi0WuQviXeE0eM6MsAg], [P], s[STARTED]: Failed to execute [org.elasticsearch.action.search.SearchRequest@3989d22b] lastShard [true]
org.elasticsearch.search.query.QueryPhaseExecutionException: [myindex3][2]: query[filtered(_all:*startup _all:maroc*)->cache(_type:tweecoms3)],from[0],size[10000000]: Query Failed [Failed to execute main query]
	at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:163)
	at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:286)
	at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:297)
	at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:231)
	at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:228)
	at org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:559)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
	at org.apache.lucene.util.PriorityQueue.<init>(PriorityQueue.java:56)
	at org.apache.lucene.search.HitQueue.<init>(HitQueue.java:64)
	at org.apache.lucene.search.TopScoreDocCollector.<init>(TopScoreDocCollector.java:275)
	at org.apache.lucene.search.TopScoreDocCollector.<init>(TopScoreDocCollector.java:37)
	at org.apache.lucene.search.TopScoreDocCollector$OutOfOrderTopScoreDocCollector.<init>(TopScoreDocCollector.java:135)
	at org.apache.lucene.search.TopScoreDocCollector$OutOfOrderTopScoreDocCollector.<init>(TopScoreDocCollector.java:133)
	at org.apache.lucene.search.TopScoreDocCollector.create(TopScoreDocCollector.java:262)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:490)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
	at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:157)
	... 8 more
[2015-11-14 20:03:11,764][DEBUG][action.search.type       ] [Pixx] [myindex3][0], node[x9gOi0WuQviXeE0eM6MsAg], [P], s[STARTED]: Failed to execute [org.elasticsearch.action.search.SearchRequest@3989d22b] lastShard [true]
org.elasticsearch.search.query.QueryPhaseExecutionException: [myindex3][0]: query[filtered(_all:*startup _all:maroc*)->cache(_type:tweecoms3)],from[0],size[10000000]: Query Failed [Failed to execute main query]
	at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:163)
	at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:286)
	at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:297)
	at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:231)
	at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:228)
	at org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:559)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
	at org.apache.lucene.util.FixedBitSet.<init>(FixedBitSet.java:187)
	at org.elasticsearch.common.lucene.docset.DocIdSets.toCacheable(DocIdSets.java:81)
	at org.elasticsearch.index.cache.filter.weighted.WeightedFilterCache$FilterCacheFilterWrapper.getDocIdSet(WeightedFilterCache.java:177)
	at

how i can resolve this problem i have dedicate server contains just a Intel Xeon W3520 c 4/8 t + 2.66 GHz RAM 32 GB ECC under Debian

2

Set size to 10 or 100.

Or use scan and scroll if you need to extract data.

Also don't use wildcard it's too slow.

1 Like