Not sure if this is the right user group, but here goes:
I'm planning to use ElasticSearch.net as the client for connecting to my ES
cluster. I have one question I haven't been able to find the answer to. I
know that the ConnectionPool feature can check if nodes fail, but can the
client also ensure that data is written to the right shard, or does it
simply use round robin to connect?
Example:
A document "1234" is created
Based on the current number of shards, the document should be put on node
6 and a replica on node 11.
Is the request sent directly to node 6, or is it sent to a random node
which will then forward the request to the right servers?
When a index request comes , based on the ID of the request , a hash
function computes the shard to which the request has to be routed.
Hence we are achieving a load balancing based on this procedure. It has to
be also noted that the routing key can be controlled.
Sniffing - While creating a client , you can use the sniffing feature for
elasticsearch to determine all the nodes and use them all in a load
balanced way.
Not sure if this is the right user group, but here goes:
I'm planning to use ElasticSearch.net as the client for connecting to my
ES cluster. I have one question I haven't been able to find the answer to.
I know that the ConnectionPool feature can check if nodes fail, but can the
client also ensure that data is written to the right shard, or does it
simply use round robin to connect?
Example:
A document "1234" is created
Based on the current number of shards, the document should be put on
node 6 and a replica on node 11.
Is the request sent directly to node 6, or is it sent to a random node
which will then forward the request to the right servers?
Alright, so the clients don't have the knowledge to route the request to
the right server. This would typically yields an extra request from the
receiving node to the node that stores the data, but on the other hand it
reduces the complexity of the client.
Can anybody confirm that this is the recommended strategy? In other
databases like Couchbase, the client knows exactly which node to route the
request to, based on a shard map that is automatically updated on regular
time intervals.
Routing - Elasticsearch Platform — Find real-time answers at scale | Elastic
When a index request comes , based on the ID of the request , a hash
function computes the shard to which the request has to be routed.
Hence we are achieving a load balancing based on this procedure. It has to
be also noted that the routing key can be controlled.
Sniffing - While creating a client , you can use the sniffing feature for
elasticsearch to determine all the nodes and use them all in a load
balanced way.
Not sure if this is the right user group, but here goes:
I'm planning to use ElasticSearch.net as the client for connecting to my
ES cluster. I have one question I haven't been able to find the answer to.
I know that the ConnectionPool feature can check if nodes fail, but can the
client also ensure that data is written to the right shard, or does it
simply use round robin to connect?
Example:
A document "1234" is created
Based on the current number of shards, the document should be put on
node 6 and a replica on node 11.
Is the request sent directly to node 6, or is it sent to a random node
which will then forward the request to the right servers?
Yes, all REST clients (that is to exclude the native Java clients) do not
participate in the cluster and therefor don't have and shouldn't have any
knowledge of the cluster. The issue a request to a node in the cluster
which then may or may not reroute it to 1 or more nodes in an efficient
manner using the cluster meta-data it has stored locally. This is by design
and has many optimizations in place as well, and also allows you as a
client to use round-robin or make requests to a load balancer.
Alright, so the clients don't have the knowledge to route the request to
the right server. This would typically yields an extra request from the
receiving node to the node that stores the data, but on the other hand it
reduces the complexity of the client.
Can anybody confirm that this is the recommended strategy? In other
databases like Couchbase, the client knows exactly which node to route the
request to, based on a shard map that is automatically updated on regular
time intervals.
Routing - Elasticsearch Platform — Find real-time answers at scale | Elastic
When a index request comes , based on the ID of the request , a hash
function computes the shard to which the request has to be routed.
Hence we are achieving a load balancing based on this procedure. It has
to be also noted that the routing key can be controlled.
Sniffing - While creating a client , you can use the sniffing feature for
elasticsearch to determine all the nodes and use them all in a load
balanced way.
Not sure if this is the right user group, but here goes:
I'm planning to use ElasticSearch.net as the client for connecting to my
ES cluster. I have one question I haven't been able to find the answer to.
I know that the ConnectionPool feature can check if nodes fail, but can the
client also ensure that data is written to the right shard, or does it
simply use round robin to connect?
Example:
A document "1234" is created
Based on the current number of shards, the document should be put on
node 6 and a replica on node 11.
Is the request sent directly to node 6, or is it sent to a random node
which will then forward the request to the right servers?
Yes, all REST clients (that is to exclude the native Java clients) do not
participate in the cluster and therefor don't have and shouldn't have any
knowledge of the cluster. The issue a request to a node in the cluster
which then may or may not reroute it to 1 or more nodes in an efficient
manner using the cluster meta-data it has stored locally. This is by design
and has many optimizations in place as well, and also allows you as a
client to use round-robin or make requests to a load balancer.
Alright, so the clients don't have the knowledge to route the request to
the right server. This would typically yields an extra request from the
receiving node to the node that stores the data, but on the other hand it
reduces the complexity of the client.
Can anybody confirm that this is the recommended strategy? In other
databases like Couchbase, the client knows exactly which node to route the
request to, based on a shard map that is automatically updated on regular
time intervals.
Routing - Elasticsearch Platform — Find real-time answers at scale | Elastic
When a index request comes , based on the ID of the request , a hash
function computes the shard to which the request has to be routed.
Hence we are achieving a load balancing based on this procedure. It has
to be also noted that the routing key can be controlled.
Sniffing - While creating a client , you can use the sniffing feature
for elasticsearch to determine all the nodes and use them all in a load
balanced way.
Not sure if this is the right user group, but here goes:
I'm planning to use ElasticSearch.net as the client for connecting to
my ES cluster. I have one question I haven't been able to find the answer
to. I know that the ConnectionPool feature can check if nodes fail, but can
the client also ensure that data is written to the right shard, or does it
simply use round robin to connect?
Example:
A document "1234" is created
Based on the current number of shards, the document should be put on
node 6 and a replica on node 11.
Is the request sent directly to node 6, or is it sent to a random
node which will then forward the request to the right servers?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.