I have a question about the cluster rebalancing. Let's say I use zone
identifiers to control how the shards are spread across the cluster.
What will happen when one of the nodes with the specified zone
identifier dies ? Will the shards and replicas be rebalanced to nodes
with the same zone identifier or can they be placed elsewhere on the
cluster ? If they will be placed elsewhere, is there a way to control
the placement ?
What do you mean by zone identifiers? You mean the sample we have with a node attribute called zone? Thats just a sample of the name of the node attribute, it can be anything you want.
On Friday, February 10, 2012 at 6:20 PM, Rafał Kuć wrote:
Hi,
I have a question about the cluster rebalancing. Let's say I use zone
identifiers to control how the shards are spread across the cluster.
What will happen when one of the nodes with the specified zone
identifier dies ? Will the shards and replicas be rebalanced to nodes
with the same zone identifier or can they be placed elsewhere on the
cluster ? If they will be placed elsewhere, is there a way to control
the placement ?
Yes, that was what I was talking about. I know zone identifier is just
an example of an attribute, just wanted to make reference to the
documentation. And yes, I'm wondering if Elasticsearch will use those
information in case one or more nodes dies or it will put the shard/
replica on a node we can't predict.
What do you mean by zone identifiers? You mean the sample we have with a node attribute called zone? Thats just a sample of the name of the node attribute, it can be anything you want.
On Friday, February 10, 2012 at 6:20 PM, Rafał Kuć wrote:
Hi,
I have a question about the cluster rebalancing. Let's say I use zone
identifiers to control how the shards are spread across the cluster.
What will happen when one of the nodes with the specified zone
identifier dies ? Will the shards and replicas be rebalanced to nodes
with the same zone identifier or can they be placed elsewhere on the
cluster ? If they will be placed elsewhere, is there a way to control
the placement ?
Not sure I follow what you mean by placing a shard on a node you can't predict. The logic is simple, if its not forced awareness, then allocation happen as usual, with balancing across awareness attributes values/groups, but its not forced. You can set it to a forced awareness which means that it will not allocate copies intended to go to a specific awareness attribute value on another one (we need to know all the values for that node attribute in order to make this call, though you can change it live using the cluster update settings API).
On Sunday, February 12, 2012 at 8:25 PM, Rafał Kuć wrote:
Hi,
Yes, that was what I was talking about. I know zone identifier is just
an example of an attribute, just wanted to make reference to the
documentation. And yes, I'm wondering if Elasticsearch will use those
information in case one or more nodes dies or it will put the shard/
replica on a node we can't predict.
What do you mean by zone identifiers? You mean the sample we have with a node attribute called zone? Thats just a sample of the name of the node attribute, it can be anything you want.
On Friday, February 10, 2012 at 6:20 PM, Rafał Kuć wrote:
Hi,
I have a question about the cluster rebalancing. Let's say I use zone
identifiers to control how the shards are spread across the cluster.
What will happen when one of the nodes with the specified zone
identifier dies ? Will the shards and replicas be rebalanced to nodes
with the same zone identifier or can they be placed elsewhere on the
cluster ? If they will be placed elsewhere, is there a way to control
the placement ?
Shay, that was what I was talking about - so if we force awareness, we
can be sure, that Elasticsearch will place the shards on a node with
the given attribute. Once again thanks for the answer.
Not sure I follow what you mean by placing a shard on a node you can't predict. The logic is simple, if its not forced awareness, then allocation happen as usual, with balancing across awareness attributes values/groups, but its not forced. You can set it to a forced awareness which means that it will not allocate copies intended to go to a specific awareness attribute value on another one (we need to know all the values for that node attribute in order to make this call, though you can change it live using the cluster update settings API).
On Sunday, February 12, 2012 at 8:25 PM, Rafał Kuć wrote:
Hi,
Yes, that was what I was talking about. I know zone identifier is just
an example of an attribute, just wanted to make reference to the
documentation. And yes, I'm wondering if Elasticsearch will use those
information in case one or more nodes dies or it will put the shard/
replica on a node we can't predict.
What do you mean by zone identifiers? You mean the sample we have with a node attribute called zone? Thats just a sample of the name of the node attribute, it can be anything you want.
On Friday, February 10, 2012 at 6:20 PM, Rafał Kuć wrote:
Hi,
I have a question about the cluster rebalancing. Let's say I use zone
identifiers to control how the shards are spread across the cluster.
What will happen when one of the nodes with the specified zone
identifier dies ? Will the shards and replicas be rebalanced to nodes
with the same zone identifier or can they be placed elsewhere on the
cluster ? If they will be placed elsewhere, is there a way to control
the placement ?
Still not sure I got your question then :). There are two aspects. If you want to make sure that an index (with all its shards and copies of them) will be allocated to specific nodes, then you can use allocation filtering.
If you are talking about a shard and its different copies (replicas), then yes, forced allocation awareness will mean that elasticsearch will only place the "average" amount of copies of a shard on each forced awareness attribute value (since a shard can have several copies).
On Monday, February 13, 2012 at 12:16 AM, Rafał Kuć wrote:
Shay, that was what I was talking about - so if we force awareness, we
can be sure, that Elasticsearch will place the shards on a node with
the given attribute. Once again thanks for the answer.
Not sure I follow what you mean by placing a shard on a node you can't predict. The logic is simple, if its not forced awareness, then allocation happen as usual, with balancing across awareness attributes values/groups, but its not forced. You can set it to a forced awareness which means that it will not allocate copies intended to go to a specific awareness attribute value on another one (we need to know all the values for that node attribute in order to make this call, though you can change it live using the cluster update settings API).
On Sunday, February 12, 2012 at 8:25 PM, Rafał Kuć wrote:
Hi,
Yes, that was what I was talking about. I know zone identifier is just
an example of an attribute, just wanted to make reference to the
documentation. And yes, I'm wondering if Elasticsearch will use those
information in case one or more nodes dies or it will put the shard/
replica on a node we can't predict.
What do you mean by zone identifiers? You mean the sample we have with a node attribute called zone? Thats just a sample of the name of the node attribute, it can be anything you want.
On Friday, February 10, 2012 at 6:20 PM, Rafał Kuć wrote:
Hi,
I have a question about the cluster rebalancing. Let's say I use zone
identifiers to control how the shards are spread across the cluster.
What will happen when one of the nodes with the specified zone
identifier dies ? Will the shards and replicas be rebalanced to nodes
with the same zone identifier or can they be placed elsewhere on the
cluster ? If they will be placed elsewhere, is there a way to control
the placement ?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.