Auto_expand_replicas and shard allocation filtering

Elastic Stack Elasticsearch
1

Hello, If I configure the nodes in my cluster to have a node attribute
'node.zone' with values "zone1", "zone1", etc
and then configure an index as follows:

curl -XPUT localhost:9200/test/_settings -d '{
"index.routing.allocation.include.zone" : "zone1",
"index.auto_expand_replicas" : "0-all"
}'

Will elasticsearch auto expand the replica shards for index 'test' only
within zone1 or will the auto_expand_replicas option ignore the shard
allocation directives and auto replicate across the entire cluster?

Regards
Mauri

--

2

Hello, Decided to do a test to investigate this.

With an index configured with auto_expand_replicas="0-all" the cluster will
try to allocate one primary and ({number nodes} - 1) replicas for each
shard, ie a copy of each shard on each node.

However, with "index.routing.allocation.include.zone"="zone1" the cluster
is blocked from allocating a shard (either primary or replica) to any nodes
that are not configured with a zone attribute set to "zone1", ie
"node.zone=zone1" in the elasticsearch.yaml config file. So if a cluster
has 3 nodes in "zone1" and 3 nodes in "zone2" it will allocate 3 shards
(primary + 2 x replicas) to the "zone1" nodes and mark an additional 3
replicas as unassigned. I observed this using the elasticsearch head
utility.

So the allocation behaviour is as desired, ie auto expand replicas to all
nodes with a specific zone attribute.
I do not know if the unassigned replicas will cause problems for the
cluster.
Can anyone add to this?

Below is the HTTP request I used to create the index

Request ---------------------------------------------
POST /test8 HTTP/1.0
Host: localhost:9200
Content-Type: application/json
Content-Length: 178

{ "settings" : { "index" : { "number_of_shards" : 1,
"auto_expand_replicas" : "0-all", "routing" : {"allocation" :
{"include" : {"zone" : "zone1"}}} } }}
Response --------------------------------------------
HTTP/1.0 200 OK
Content-Type: application/json; charset=UTF-8
Content-Length: 31

{"ok":true,"acknowledged":true}
End ----------------------------------------------

Regards
Mauri

--

3

We are experiencing this as well. The Unassigned shards in this case (when
auto-expand-replica is set to some value and the index has include/exclude
stuff set) will make your cluster status be in RED state, which is bad.

I opened a ticket for this here:

On Wednesday, September 5, 2012 5:58:11 PM UTC+3, Mauri wrote:

Hello, Decided to do a test to investigate this.

With an index configured with auto_expand_replicas="0-all" the cluster
will try to allocate one primary and ({number nodes} - 1) replicas for each
shard, ie a copy of each shard on each node.

However, with "index.routing.allocation.include.zone"="zone1" the cluster
is blocked from allocating a shard (either primary or replica) to any nodes
that are not configured with a zone attribute set to "zone1", ie
"node.zone=zone1" in the elasticsearch.yaml config file. So if a cluster
has 3 nodes in "zone1" and 3 nodes in "zone2" it will allocate 3 shards
(primary + 2 x replicas) to the "zone1" nodes and mark an additional 3
replicas as unassigned. I observed this using the elasticsearch head
utility.

So the allocation behaviour is as desired, ie auto expand replicas to all
nodes with a specific zone attribute.
I do not know if the unassigned replicas will cause problems for the
cluster.
Can anyone add to this?

Below is the HTTP request I used to create the index

Request ---------------------------------------------
POST /test8 HTTP/1.0
Host: localhost:9200
Content-Type: application/json
Content-Length: 178

{ "settings" : { "index" : { "number_of_shards" : 1,
"auto_expand_replicas" : "0-all", "routing" : {"allocation" :
{"include" : {"zone" : "zone1"}}} } }}
Response --------------------------------------------
HTTP/1.0 200 OK
Content-Type: application/json; charset=UTF-8
Content-Length: 31

{"ok":true,"acknowledged":true}
End ----------------------------------------------

Regards
Mauri

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.