Help with cluster.routing.allocation.enable Property

Elastic Stack Elasticsearch
1

Hi folks,

Our requirement is as follows:

We have our ES cluster running on aws instances where nodes can come and
go. We would like to disable Shard reallocation on cluster in Normal state
so that Node going down should not rebalance the shards.(Just to avoid
unnecessary Network i/o).

When node comes back up (within few minutes), we would like to enable it so
that new node will get back it's own shards and after rebalancing is
finished, we would like to Disable Shard reallocation again.

We would like this to be an automated process rather than manual one.

I have already taken a look at following stackoverflow question/answer
http://stackoverflow.com/questions/16231614/quick-recovery-after-node-restart-in-elasticsearch
but I am not sure if it can work in automated way.

curl -XPUT localhost:9200/_cluster/settings -d '{"transient":{"cluster.routing.allocation.enable": "none"}}'
/etc/init.d/elasticsearch restart
curl -XPUT localhost:9200/_cluster/settings -d '{"transient":{"cluster.routing.allocation.enable": "all"}}'

So I would like to know, is there any way to Listen to the event which says
"Shard Rebalancing is finished" (by adding a new listener) and then
disable cluster.routing.allocation.enable Property ?

-- Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/cee6bef8-25cf-4175-ac53-f2734d44ea76%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

The cluster state will be yellow while shards are being rebalanced, so you
can just execute a cluster healt request and wait for green status:

You can also poll the same API and wait until all the moving shards
(relocating_shards, initializing_shards, unassigned_shards) are 0.

Cheers,

Ivan

On Fri, Jun 6, 2014 at 11:03 AM, sagarl sagarit2@gmail.com wrote:

Hi folks,

Our requirement is as follows:

We have our ES cluster running on aws instances where nodes can come and
go. We would like to disable Shard reallocation on cluster in Normal state
so that Node going down should not rebalance the shards.(Just to avoid
unnecessary Network i/o).

When node comes back up (within few minutes), we would like to enable it
so that new node will get back it's own shards and after rebalancing is
finished, we would like to Disable Shard reallocation again.

We would like this to be an automated process rather than manual one.

I have already taken a look at following stackoverflow question/answer
http://stackoverflow.com/questions/16231614/quick-recovery-after-node-restart-in-elasticsearch
but I am not sure if it can work in automated way.

curl -XPUT localhost:9200/_cluster/settings -d '{"transient":{"cluster.routing.allocation.enable": "none"}}'
/etc/init.d/elasticsearch restart
curl -XPUT localhost:9200/_cluster/settings -d '{"transient":{"cluster.routing.allocation.enable": "all"}}'

So I would like to know, is there any way to Listen to the event which
says "Shard Rebalancing is finished" (by adding a new listener) and then
disable cluster.routing.allocation.enable Property ?

-- Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/cee6bef8-25cf-4175-ac53-f2734d44ea76%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/cee6bef8-25cf-4175-ac53-f2734d44ea76%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCXbKF%3DDCig3xhe48yb40Tb88tLXRH2kNk3w4OxJq62aQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

sagarl wrote:

We have our ES cluster running on aws instances where nodes can
come and go. We would like to disable Shard reallocation on
cluster in Normal state so that Node going down should not
rebalance the shards.(Just to avoid unnecessary Network i/o).

If your shard topology is relatively static, you could just leave
allocation disabled (or set it to new_primaries for the occasional
index creation). But honestly you're trying to micro-manage a
process that ES does pretty well. There's nothing wrong with
having a yellow cluster. If you're concerned about availability
you should add replicas.

Drew

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/m2y4x9wwax.fsf%40mid.raines.me.
For more options, visit https://groups.google.com/d/optout.

4

Thanks Ivan for the suggestions, I'll try to make use of them.

Drew, Sorry I din't get the part when you said "There's nothing wrong with
having a yellow cluster" And yes I agree ES does a good job of rebalancing
but just think of a case where you have close to few hundred GBs of data
per shard which will move around the cluster even though the new node takes
only few minutes to come back up. Hence, looking to avoid unnecessary
network i/O.

-- Sagar

On Friday, June 6, 2014 2:41:37 PM UTC-7, Drew Raines wrote:

sagarl wrote:

We have our ES cluster running on aws instances where nodes can
come and go. We would like to disable Shard reallocation on
cluster in Normal state so that Node going down should not
rebalance the shards.(Just to avoid unnecessary Network i/o).

If your shard topology is relatively static, you could just leave
allocation disabled (or set it to new_primaries for the occasional
index creation). But honestly you're trying to micro-manage a
process that ES does pretty well. There's nothing wrong with
having a yellow cluster. If you're concerned about availability
you should add replicas.

Drew

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/386d674e-52b8-4ed1-95ce-6efcf03718c0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

5

sagarl wrote:

Drew, Sorry I din't get the part when you said "There's nothing
wrong with having a yellow cluster" And yes I agree ES does a
good job of rebalancing but just think of a case where you have
close to few hundred GBs of data per shard which will move
around the cluster even though the new node takes only few
minutes to come back up. Hence, looking to avoid unnecessary
network i/O.

If you were referring to /planned/ node outages, then I agree. I
thought you were trying to reinvent schemes for dealing with the
unpredictable. Apologies for misinterpreting!

Drew

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/m2ppilwicj.fsf%40mid.raines.me.
For more options, visit https://groups.google.com/d/optout.