Got many thread waiting in ES TransportClient

nodexy · January 29, 2016, 3:26pm

I got this issue when invoke es search api in java client . When invoke search api in java by TransportClient under 500/1000/2000 concurrency press test , there are many threads waiting

Here is the whole stack dump :

gist.github.com

https://gist.github.com/node/773dc44273d0c78680ed

es-2.1.1-waiting-thread-dump.txt

"catalina-exec-35" daemon prio=10 tid=0x00007fdb08020800 nid=0x887a waiting on condition [0x00007fdabedeb000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000007f0ca71c8> (a org.elasticsearch.common.util.concurrent.BaseFuture$Sync)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
        at org.elasticsearch.common.util.concurrent.BaseFuture$Sync.get(BaseFuture.java:276)
        at org.elasticsearch.common.util.concurrent.BaseFuture.get(BaseFuture.java:116)

This file has been truncated. show original

and here is my code to search
SearchResponse response = esClient .prepareSearch(esManager.getIndexName()) .setTypes(esManager.getIndexType()) .setSearchType(SearchType.DFS_QUERY_THEN_FETCH) .setQuery(mmQuery).setPostFilter(andSearchFilter) .setTrackScores(true) .setMinScore(MyConst.SEARCH_ES_MINSCORE) .addSort(SortBuilders.scoreSort()) .addSort("official", SortOrder.DESC) .addSort("downloadCount", SortOrder.DESC) .setFrom(pageFrom).setSize(pageSize) .setExplain(true).execute().actionGet();

mmQuery is a MultiMatchQueryBuilder with two fileds , and andSearchFilter is a BoolQueryBuilder .

Any suggestion here ?