Repeated NoNodeAvailableExceptions Exceptions

(Ben) #1

Hi all. I'm finding that I am getting repeated NodeDisconnectedException and NoNodeAvailableExceptions exceptions when running relatively few concurrent queries - fewer than 20.

I think I am beginning to understand the reason why but some confirmation would be most appreciated.

The default queue size is 1000 and the default number of shards per index is 5. A query against each shard uses 1 in the queue. So, for example, querying against 20 indexes with 5 shards each you would only need to run 101 concurrent queries to go over the default queue size and cause these exceptions?

Is my understanding correct?



(Lee Hinman) #2

That queue size is for the action-specific threadpool, it should not affect the
NodeDisconnectedException and NoNodeAvailableException failures that you are
seeing, those exceptions are because either a node has been disconnected and
detected with fault detection, or a node cannot be contacted.

You should check the logs for the full message for the node disconnection
exceptions and see whether they failed during communication.

(Ben) #3

Hi Lee,

Thanks for the reply. I am not seeing anything in the Elasticsearch logs on either node. All the errors are logged on the client side which I have pasted below.

I have tried a few options on the Transport client such as turning sniff on and increasing the ping and node sampler interval values but this has made no difference. Is there anything I can do to help debug the error?

I am using v5.1.2 of both Elasticsearch and the Java client. My setup consists of two virtual machines each with 2 CPUs, 4GB RAM and 2GB allocated to the JVM.


None of the configured nodes were available: [{escluster-es2}{-iqECyceSn2FBSPbg-nFvw}{F2PCjdZTTUujzj5996um6w}{}{}, {escluster-es1}{WELN6zJ8QYy40zew1Axs2w}{Zc4W94tbT7exlRHziAnd-g}{}{}]: NoNodeAvailableException[None of the configured nodes were available: [{escluster-es2}{-iqECyceSn2FBSPbg-nFvw}{F2PCjdZTTUujzj5996um6w}{}{}, {escluster-es1}{WELN6zJ8QYy40zew1Axs2w}{Zc4W94tbT7exlRHziAnd-g}{}{}]]; nested: NodeDisconnectedException[[escluster-es1][][indices:data/read/search] disconnected];
at org.elasticsearch.client.transport.TransportClientNodesService$RetryListener.onFailure(
at org.elasticsearch.action.ActionListenerResponseHandler.handleException(
at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(
at org.elasticsearch.transport.TransportService$Adapter.lambda$raiseNodeDisconnected$3(
at org.elasticsearch.common.util.concurrent.ThreadContext$
at java.util.concurrent.ThreadPoolExecutor.runWorker(
at java.util.concurrent.ThreadPoolExecutor$
Caused by: NodeDisconnectedException[[escluster-es1][][indices:data/read/search] disconnected]

(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.