Request rejected when multiple search request

Hi Guys,

I am trying to call multiple(300) elasticsearch search request Async and I got the following error. It seems like some of my requests being rejected. Anyone experience this before.
I am still new to Elasticsearch, please correct me if any wrong. Thanks

2019-08-30T16:21:32,588][DEBUG][o.e.a.s.TransportSearchAction] [Windows-PC] [data][4], node[tYdfN-D-REye4n7029yCQw], [P], s[STARTED], a[id=z4waoy-VRwuXqO3x6XOypQ]: Failed to execute [SearchRequest{searchType=QUERY_THEN_FETCH, indices=[log], indicesOptions=IndicesOptions[ignore_unavailable=false, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, allow_aliases_to_multiple_indices=true, forbid_closed_indices=true, ignore_aliases=false, ignore_throttled=true], types=, routing='null', preference='null', requestCache=null, scroll=null, maxConcurrentShardRequests=0, batchedReduceSize=512, preFilterShardSize=128, allowPartialSearchResults=true, localClusterAlias=null, getOrCreateAbsoluteStartMillis=-1, ccsMinimizeRoundtrips=true, source={"query":{"query_string":{"query":"data.id:3","fields":,"type":"best_fields","default_operator":"or","max_determinized_states":10000,"enable_position_increments":true,"fuzziness":"AUTO","fuzzy_prefix_length":0,"fuzzy_max_expansions":50,"phrase_slop":0,"analyze_wildcard":false,"escape":false,"auto_generate_synonyms_phrase_query":true,"fuzzy_transpositions":true,"boost":1.0}}}}] lastShard [true]
org.elasticsearch.transport.RemoteTransportException: [Windows-PC][127.0.0.1:9300][indices:data/read/search[phase/query]]
Caused by: org.elasticsearch.common.util.concurrent.EsRejectedExecutionException: rejected execution of org.elasticsearch.common.util.concurrent.TimedRunnable@7a92c500 on QueueResizingEsThreadPoolExecutor[name = WIN10-PC/search, queue capacity = 1000, min queue capacity = 1000, max queue capacity = 2000, frame size = 2000, targeted response rate = 1s, task execution EWMA = 41.4ms, adjustment amount = 50, org.elasticsearch.common.util.concurrent.QueueResizingEsThreadPoolExecutor@538a0253[Running, pool size = 4, active threads = 4, queued tasks = 1000, completed tasks = 5766]]
at org.elasticsearch.common.util.concurrent.EsAbortPolicy.rejectedExecution(EsAbortPolicy.java:48) ~[elasticsearch-7.1.1.jar:7.1.1]
at java.util.concurrent.ThreadPoolExecutor.reject(Unknown Source) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor.execute(Unknown Source) ~[?:?]
at org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor.execute(EsThreadPoolExecutor.java:98) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.search.SearchService.lambda$rewriteShardRequest$4(SearchService.java:1061) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.index.shard.IndexShard.awaitShardSearchActive(IndexShard.java:3024) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.search.SearchService.lambda$rewriteShardRequest$5(SearchService.java:1060) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:61) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:114) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:87) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.search.SearchService.rewriteShardRequest(SearchService.java:1076) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:354) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.action.search.SearchTransportService.lambda$registerRequestHandler$7(SearchTransportService.java:352) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.transport.TransportService.sendLocalRequest(TransportService.java:687) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.transport.TransportService.access$000(TransportService.java:73) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.transport.TransportService$3.sendRequest(TransportService.java:125) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.transport.TransportService.sendRequestInternal(TransportService.java:633) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:543) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.transport.TransportService.sendChildRequest(TransportService.java:587) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.transport.TransportService.sendChildRequest(TransportService.java:578) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.action.search.SearchTransportService.sendExecuteQuery(SearchTransportService.java:137) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.action.search.SearchQueryThenFetchAsyncAction.executePhaseOnShard(SearchQueryThenFetchAsyncAction.java:54) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.action.search.InitialSearchPhase.lambda$performPhaseOnShard$4(InitialSearchPhase.java:259) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.action.search.InitialSearchPhase.performPhaseOnShard(InitialSearchPhase.java:294) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.action.search.InitialSearchPhase.run(InitialSearchPhase.java:158) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.executePhase(AbstractSearchAsyncAction.java:166) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.start(AbstractSearchAsyncAction.java:126) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.action.search.TransportSearchAction.executeSearch(TransportSearchAction.java:514) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.action.search.TransportSearchAction.executeLocalSearch(TransportSearchAction.java:399) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.action.search.TransportSearchAction.lambda$doExecute$3(TransportSearchAction.java:211) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:61) [elasticsearch-7.1.1.jar:7.1.1]
at org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:114) ...

Yes, it looks like you are overwhelming the cluster. As it is not able to keep up the queue fills up, after which requests are rejected. Each query uses at least one slot on the queue per shard it queries so it could be that you simply have too many shards to allow all the 300 concurrent requests to be queued.

It seems like you are running a single node, but it is not clear how much resources this have. I would therefore recommend starting with a lower concurrency level and gradually increase until you find the limit of the node or simply increase the size of the cluster.

Hi Christian, thanks for explanation.
Yes, I am running this with one single node. By the way, what kind of resources you are referring to ?

With resouces I am referring to CPU, RAM and IOPS.

Currently I am running single elastic node in server machine with 2 processors and 8GB RAM.
If i run 200 search request concurrently, it's working fine. Should i configure setting for the thread pool?

How many indices and shards is each query targeting?

1 indices with 5 shards

How large is it? If less than a few Tens of GB, can you try shrinking it down to a single shard?

My index is around 25 GB. Is it needed?
Should i configure the threadpool setting in elasticsearch.yml?

thread_pool.search.queue_size:
thread_pool.search.size:
thread_pool.search.min_queue_size:
thread_pool.search.max_queue_size:
thread_pool.search.auto_queue_frame_size:
thread_pool.search.target_response_time:

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.