I have a 3 node cluster in production in 5.6 version. I agreggate the data in daily indices and sometimes my cluster become unavailable and the exception that appears is "search_phase_exception".
Reading the cluster logs i can see the following error repeatedly :
caused by: org.elasticsearch.common.util.concurrent.EsRejectedExecutionException: rejected execution of org.elasticsearch.transport.TransportService on EsThreadPoolExecutor[search, queue capacity = 1000, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor[Running, pool size = 16, active threads = 16, queued tasks = 64910, completed tasks = 67899727]]
I think this is related with threadpool search queue capacity, and the solution is in general put another node to the cluster but i think that my configuration can be optimized.
May you help me to understand this exception ?