Query routing inside of a cluster


(Savva) #1

Hi,

I have a question regarding the query routing inside of a cluster. Let say, node holds full data replication and can satisfy every search request it receives. Will it redirect/reroute queries to other nodes in the cluster in case of a high load?

The motivation is the following: I have 3 nodes distributed across availability zones; 10 indices with 1 shard and 2 replicas each.

Now, in each zone there is a group of machines that connects to the node in the same zone via the transport client. Sniffing is turned off so as to reduce network latency and avoid requests from the clients to nodes across the zones.

I'm wondering if nodes may redirect (for some reason) queries to other nodes in the cluster, even if they contain all necessary information for every request.

If this is the case, is there any way to avoid such behavior, cause it produces additional hops that affect search time?

Thank you,
Savva

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2fe2dc69-6878-4525-8754-d409ef56df8d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #2

Only nodes that hold a shard (primary or replica) of an index you specified
participate in handling queries. Replica are selected by round-robin afaik
(or maybe at random order?). Which nodes are selected by a query is
determined at once by the node that receives the query within the cluster.
It is the node the TransportClient is connected to, because the
TransportClient is not able to keep a copy of the cluster state. This proxy
node also collects the results from the shards and builds the response.
There is no forwarding or redirecting from one node to another after the
proxy node has dispatched the query down to the shard level.

Jörg

On Wed, May 14, 2014 at 7:11 PM, Savva savva.khalaman@gmail.com wrote:

Hi,

I have a question regarding the query routing inside of a cluster. Let
say, node holds full data replication and can satisfy every search request
it receives. Will it redirect/reroute queries to other nodes in the cluster
in case of a high load?

The motivation is the following: I have 3 nodes distributed across
availability zones; 10 indices with 1 shard and 2 replicas each.

Now, in each zone there is a group of machines that connects to the node
in the same zone via the transport client. Sniffing is turned off so as to
reduce network latency and avoid requests from the clients to nodes across
the zones.

I'm wondering if nodes may redirect (for some reason) queries to other
nodes in the cluster, even if they contain all necessary information for
every request.

If this is the case, is there any way to avoid such behavior, cause it
produces additional hops that affect search time?

Thank you,
Savva

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/2fe2dc69-6878-4525-8754-d409ef56df8d%40googlegroups.com
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoF%3DDS-US5nguySRYqMw1tVo%3DucbohSLmPH6woMxTHLNtw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Clinton Gormley) #3

Hi Savva

I presume you're using cluster.routing.allocation.awareness? If so, then
shards on nodes with the same node attributes are preferred:

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-cluster.html#_automatic_preference_when_searching_geting

On 14 May 2014 19:28, joergprante@gmail.com joergprante@gmail.com wrote:

Only nodes that hold a shard (primary or replica) of an index you
specified participate in handling queries. Replica are selected by
round-robin afaik (or maybe at random order?). Which nodes are selected by
a query is determined at once by the node that receives the query within
the cluster. It is the node the TransportClient is connected to, because
the TransportClient is not able to keep a copy of the cluster state. This
proxy node also collects the results from the shards and builds the
response. There is no forwarding or redirecting from one node to another
after the proxy node has dispatched the query down to the shard level.

Jörg

On Wed, May 14, 2014 at 7:11 PM, Savva savva.khalaman@gmail.com wrote:

Hi,

I have a question regarding the query routing inside of a cluster. Let
say, node holds full data replication and can satisfy every search request
it receives. Will it redirect/reroute queries to other nodes in the cluster
in case of a high load?

The motivation is the following: I have 3 nodes distributed across
availability zones; 10 indices with 1 shard and 2 replicas each.

Now, in each zone there is a group of machines that connects to the node
in the same zone via the transport client. Sniffing is turned off so as to
reduce network latency and avoid requests from the clients to nodes across
the zones.

I'm wondering if nodes may redirect (for some reason) queries to other
nodes in the cluster, even if they contain all necessary information for
every request.

If this is the case, is there any way to avoid such behavior, cause it
produces additional hops that affect search time?

Thank you,
Savva

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/2fe2dc69-6878-4525-8754-d409ef56df8d%40googlegroups.com
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoF%3DDS-US5nguySRYqMw1tVo%3DucbohSLmPH6woMxTHLNtw%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CAKdsXoF%3DDS-US5nguySRYqMw1tVo%3DucbohSLmPH6woMxTHLNtw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKS6D1PX060uRiVFePNCoRzEQbgzFhwLk-JKkq3nBp6SQQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Savva) #4

Jörg and Clinton, thank you very much for so quick answers.

Jörg, just to make sure that I got things right.
Assume, the node (that the TransportClient is connected to) holds shard of
the specified in query index and no cluster.routing.allocation.awareness is
applied. Then, this proxy node still may decide to apply the request on the
shard located on the other node (for example, to perform the round-robin on
replica)?

Clinton, I'm not using cluster.routing.allocation.awareness yet. Thank you
for the hint!

Savva

On Wednesday, May 14, 2014 10:08:24 PM UTC+3, Clinton Gormley wrote:

Hi Savva

I presume you're using cluster.routing.allocation.awareness? If so, then
shards on nodes with the same node attributes are preferred:

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-cluster.html#_automatic_preference_when_searching_geting

On 14 May 2014 19:28, joerg...@gmail.com <javascript:> <joerg...@gmail.com<javascript:>

wrote:

Only nodes that hold a shard (primary or replica) of an index you
specified participate in handling queries. Replica are selected by
round-robin afaik (or maybe at random order?). Which nodes are selected by
a query is determined at once by the node that receives the query within
the cluster. It is the node the TransportClient is connected to, because
the TransportClient is not able to keep a copy of the cluster state. This
proxy node also collects the results from the shards and builds the
response. There is no forwarding or redirecting from one node to another
after the proxy node has dispatched the query down to the shard level.

Jörg

On Wed, May 14, 2014 at 7:11 PM, Savva <savva.k...@gmail.com<javascript:>

wrote:

Hi,

I have a question regarding the query routing inside of a cluster. Let
say, node holds full data replication and can satisfy every search request
it receives. Will it redirect/reroute queries to other nodes in the cluster
in case of a high load?

The motivation is the following: I have 3 nodes distributed across
availability zones; 10 indices with 1 shard and 2 replicas each.

Now, in each zone there is a group of machines that connects to the node
in the same zone via the transport client. Sniffing is turned off so as to
reduce network latency and avoid requests from the clients to nodes across
the zones.

I'm wondering if nodes may redirect (for some reason) queries to other
nodes in the cluster, even if they contain all necessary information for
every request.

If this is the case, is there any way to avoid such behavior, cause it
produces additional hops that affect search time?

Thank you,
Savva

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/2fe2dc69-6878-4525-8754-d409ef56df8d%40googlegroups.com
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoF%3DDS-US5nguySRYqMw1tVo%3DucbohSLmPH6woMxTHLNtw%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CAKdsXoF%3DDS-US5nguySRYqMw1tVo%3DucbohSLmPH6woMxTHLNtw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/05c2c76e-6b4d-41b5-99c5-e3dd0c82a2b7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #5