# Error in Elasticseach logs

**URL:** <https://discuss.elastic.co/t/error-in-elasticseach-logs/118419>\
**Category:** Elasticsearch\
**Created:** [February 5, 2018, 9:47am UTC](https://discuss.elastic.co/t/error-in-elasticseach-logs/118419 "2018-02-05T09:47:05Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kaouther\_Mechri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kaouther_mechri/32/55329_2.png) [@Kaouther\_Mechri](https://discuss.elastic.co/u/Kaouther_Mechri)\
**Post date:** [February 5, 2018, 9:47am UTC](https://discuss.elastic.co/t/error-in-elasticseach-logs/118419/1 "2018-02-05T09:47:05Z")

</div>

Hello ,  
When I access to dashbord in Kibana I have this message in elasticsearch logs .  
did you encouter such problems ?  
thanks  
Sc2w4wt3q-7cZg]: Failed to execute [SearchRequest{searchType=QUERY\_THEN\_FETCH, indices=[\*], indicesOptions=IndicesOptions[id=39, ignore\_unavailable=true, allow\_no\_indices=true, expand\_wildcards\_open=true, expand\_wildcards\_closed=false, allow\_aliases\_to\_multiple\_indices=true, forbid\_closed\_indices=true, ignore\_aliases=false], types=[], routing='null', preference='null', requestCache=null, scroll=null, maxConcurrentShardRequests=5, batchedReduceSize=512, preFilterShardSize=18, source={"size":0,"query":{"bool":{"must":[{"range":{"@timestamp":{"from":1517780215222,"to":1517823415222,"include\_lower":true,"include\_upper":true,"format":"epoch\_millis","boost":1.0}}},{"bool":{"must":[{"query\_string":{"query":"beat.name:"vm-mule-25"","fields":[],"type":"best\_fields","default\_operator":"or","max\_determinized\_states":10000,"enable\_position\_increments":true,"fuzziness":"AUTO","fuzzy\_prefix\_length":0,"fuzzy\_max\_expansions":50,"phrase\_slop":0,"escape":false,"auto\_generate\_synonyms\_phrase\_query":true,"fuzzy\_transpositions":true,"boost":1.0}}],"adjust\_pure\_negative":true,"boost":1.0}}],"adjust\_pure\_negative":true,"boost":1.0}},"aggregations":{"993acf30-1b19-11e7-b09e-037021c4f8df":{"filter":{"match\_all":{"boost":1.0}},"aggregations":{"timeseries":{"date\_histogram":{"field":"@timestamp","time\_zone":"Europe/Berlin","interval":"300s","offset":0,"order":{"\_key":"asc"},"keyed":false,"min\_doc\_count":0,"extended\_bounds":{"min":1517780215222,"max":1517823415222}},"aggregations":{"993acf31-1b19-11e7-b09e-037021c4f8df":{"avg":{"field":"system.cpu.system.pct"}}}}}}}}}] lastShard [true]  
org.elasticsearch.transport.RemoteTransportException: [elacticsearch.unealgroupe.fr][192.168.9.160:9300][indices:data/read/search[phase/query]]  
Caused by: org.elasticsearch.common.util.concurrent.EsRejectedExecutionException: rejected execution of org.elasticsearch.common.util.concurrent.TimedRunnable@2e110ea4 on QueueResizingEsThreadPoolExecutor[search, queue capacity = 1000, min queue capacity = 1000, max queue capacity = 1000, frame size = 2000, targeted response rate = 1s, task execution EWMA = 721nanos, adjustment amount = 50, QueueResizingEsThreadPoolExecutor[search, queue capacity = 1000, org.elasticsearch.common.util.concurrent.QueueResizingEsThreadPoolExecutor@4420358a[Running, pool size = 4, active threads = 4, queued tasks = 1084, completed tasks = 3885]]]  
at org.elasticsearch.common.util.concurrent.EsAbortPolicy.rejectedExecution(EsAbortPolicy.java:48) ~[elasticsearch-6.1.1.jar:6.1.1]  
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823) ~[?:1.8.0\_131]  
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369) ~[?:1.8.0\_131]  
at org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor.doExecute(EsThreadPoolExecutor.java:94) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.common.util.concurrent.QueueResizingEsThreadPoolExecutor.doExecute(QueueResizingEsThreadPoolExecutor.java:93) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor.execute(EsThreadPoolExecutor.java:89) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.search.SearchService.lambda$rewriteShardRequest$0(SearchService.java:988) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:60) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:113) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:86) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.search.SearchService.rewriteShardRequest(SearchService.java:986) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:306) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:376) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:373) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:66) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.transport.TransportService.sendLocalRequest(TransportService.java:646) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.transport.TransportService.access$000(TransportService.java:74) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.transport.TransportService$3.sendRequest(TransportService.java:135) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.transport.TransportService.sendRequestInternal(TransportService.java:594) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:514) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.transport.TransportService.sendChildRequest(TransportService.java:554) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.transport.TransportService.sendChildRequest(TransportService.java:545) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.action.search.SearchTransportService.sendExecuteQuery(SearchTransportService.java:156) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.action.search.SearchQueryThenFetchAsyncAction.executePhaseOnShard(SearchQueryThenFetchAsyncAction.java:52) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.action.search.InitialSearchPhase.performPhaseOnShard(InitialSearchPhase.java:208) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.action.search.InitialSearchPhase.run(InitialSearchPhase.java:153) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.action.search.AbstractSearchAsyncAction.executePhase(AbstractSearchAsyncAction.java:146) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.action.search.AbstractSearchAsyncAction.start(AbstractSearchAsyncAction.java:116) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.action.search.TransportSearchAction$1.run(TransportSearchAction.java:393) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.action.search.AbstractSearchAsyncAction.executePhase(AbstractSearchAsyncAction.java:146) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:140) ~[elasticsearch-6.1.1.jar:6.1.1]  
at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:243) ~[elasticsearch-6.1.1.jar:6.1.1]  
....

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 5, 2018, 10:10am UTC](https://discuss.elastic.co/t/error-in-elasticseach-logs/118419/2 "2018-02-05T10:10:20Z")

</div>

Your node is overloaded and is rejecting request to make sure it can stay afloat. You should add more nodes, more resources and/or both/

---

<div class="post-metadata">

**Author:** ![andrew\_09](https://avatars.discourse-cdn.com/v4/letter/a/65b543/32.png) [@andrew\_09](https://discuss.elastic.co/u/andrew_09)\
**Post date:** [February 5, 2018, 10:19am UTC](https://discuss.elastic.co/t/error-in-elasticseach-logs/118419/3 "2018-02-05T10:19:32Z")

</div>

How many data nodes do you have in your cluster ?  
How many indices/size are your backing up each day ?

---

<div class="post-metadata">

**Author:** ![Kaouther\_Mechri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kaouther_mechri/32/55329_2.png) [@Kaouther\_Mechri](https://discuss.elastic.co/u/Kaouther_Mechri)\
**Post date:** [February 5, 2018, 10:29am UTC](https://discuss.elastic.co/t/error-in-elasticseach-logs/118419/4 "2018-02-05T10:29:44Z")

</div>

It is a one node for testing .  
how can I check the number of indexes .

Please could you suggest me a documentation for production architecture

thanks you

---

<div class="post-metadata">

**Author:** ![Kaouther\_Mechri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kaouther_mechri/32/55329_2.png) [@Kaouther\_Mechri](https://discuss.elastic.co/u/Kaouther_Mechri)\
**Post date:** [February 5, 2018, 10:30am UTC](https://discuss.elastic.co/t/error-in-elasticseach-logs/118419/5 "2018-02-05T10:30:30Z")

</div>

In kibana I get this message  
Visualize: Request to Elasticsearch failed: {"error":{"root\_cause":[],"type":"search\_phase\_execution\_exception","reason":"","phase":"fetch","grouped":true,"failed\_shards":[],"caused\_by":{"type":"es\_rejected\_execution\_exception","reason":"rejected execution of org.elasticsearch.common.util.concurrent.TimedRunnable@77b0e659 on QueueResizingEsThreadPoolExecutor[search, queue capacity = 1000, min queue capacity = 1000, max queue capacity = 1000, frame size = 2000, targeted response rate = 1s, task execution EWMA = 66.5micros, adjustment amount = 50, QueueResizingEsThreadPoolExecutor[search, queue capacity = 1000, org.elasticsearch.common.util.concurrent.QueueResizingEsThreadPoolExecutor@4420358a[Running, pool size = 4, active threads = 4, queued tasks = 1012, completed tasks = 108762]]]"}},"status":503}

less

---

<div class="post-metadata">

**Author:** ![andrew\_09](https://avatars.discourse-cdn.com/v4/letter/a/65b543/32.png) [@andrew\_09](https://discuss.elastic.co/u/andrew_09)\
**Post date:** [February 5, 2018, 11:41am UTC](https://discuss.elastic.co/t/error-in-elasticseach-logs/118419/6 "2018-02-05T11:41:35Z")

</div>

You can find all the information you need here: [https://www.elastic.co/guide/index.html](https://www.elastic.co/guide/index.html)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 5, 2018, 11:42am UTC](https://discuss.elastic.co/t/error-in-elasticseach-logs/118419/7 "2018-03-05T11:42:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
