OutOfMemory Error for Priority Queue for "Sync" search

(komaln) #1

Hi all,

I am using elasticsearch to search through my logs. The data I am searching through is really large. I am using sync query using the function "query_then_fetch".
But when I set large "size" (more than 10,000) using "builder.setSize" I see following OutOfMemory error:

Exception in thread "elasticsearch[search]-pool-3-thread-1" java.lang.OutOfMemoryError: Requested array size e
at org.apache.lucene.util.PriorityQueue.initialize(PriorityQueue.java:108)
at org.apache.lucene.search.ShardFieldDocSortedHitQueue.(ShardFieldDocSortedHitQueue.java:48)
at org.elasticsearch.search.controller.SearchPhaseController.sortDocs(SearchPhaseController.java:189)

Is there any way to know how much "size" (builder.setSize) can be set before we get this OutOfMemory error.

Thanks and Regards,

(system) #2