Rolling restart

Elastic Stack Elasticsearch
1

I'm maintaining a small cluster of 9 nodes, and was trying to perform
rolling restart as outlined
here: http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_rolling_restarts.html#_rolling_restarts

The problem is that after I disable reallocation and restart a single node,
it appears it looses all its shards indefinitely (until I turn back
reallocation). So if I do this for all nodes in the cluster I'll run out of
primary shards at some point.

I have an upstart task for Elasticsearch, so I stopped nodes with that (it
sends SIGTERM). I tried the shutdown API but it did have the same effect --
after node joins the cluster, it doesn't own any shards, and that doesn't
change if I wait for a while.

Am I doing something wrong?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c9eef95e-d7cf-4278-a99f-89d9ab878791%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

You have to reenable allocation after the node comes back and wait for the
shards to initialize there.

On Fri, Dec 19, 2014 at 3:23 PM, iskren.chernev@gmail.com wrote:

I'm maintaining a small cluster of 9 nodes, and was trying to perform
rolling restart as outlined here:
Elasticsearch Platform — Find real-time answers at scale | Elastic

The problem is that after I disable reallocation and restart a single
node, it appears it looses all its shards indefinitely (until I turn back
reallocation). So if I do this for all nodes in the cluster I'll run out of
primary shards at some point.

I have an upstart task for Elasticsearch, so I stopped nodes with that (it
sends SIGTERM). I tried the shutdown API but it did have the same effect --
after node joins the cluster, it doesn't own any shards, and that doesn't
change if I wait for a while.

Am I doing something wrong?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c9eef95e-d7cf-4278-a99f-89d9ab878791%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c9eef95e-d7cf-4278-a99f-89d9ab878791%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd2zKhhiDpwH%2BWj4SoQYP5B6C5seET%2BBtTYCDM%2B-3rS%3D0A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

On Friday, December 19, 2014 12:31:33 PM UTC-8, Nikolas Everett wrote:

You have to reenable allocation after the node comes back and wait for the
shards to initialize there.

So this means the tutorial is wrong (current version):

  1. Disable allocation
  2. stop node
  3. ...
  4. start node
  5. Repeat 3-5 for the rest of your nodes
  6. Re-enable shard allocation using ...

It should be:

  1. disable allocation
  2. stop node
  3. ...
  4. start node
  5. enable allocation
  6. repeat steps 2-6 for the rest of your nodes

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/066a7a52-c011-44c3-b630-953c82ab4818%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

I believe so.

On Fri, Dec 19, 2014 at 3:39 PM, iskren.chernev@gmail.com wrote:

On Friday, December 19, 2014 12:31:33 PM UTC-8, Nikolas Everett wrote:

You have to reenable allocation after the node comes back and wait for
the shards to initialize there.

So this means the tutorial is wrong (current version):

  1. Disable allocation
  2. stop node
  3. ...
  4. start node
  5. Repeat 3-5 for the rest of your nodes
  6. Re-enable shard allocation using ...

It should be:

  1. disable allocation
  2. stop node
  3. ...
  4. start node
  5. enable allocation
  6. repeat steps 2-6 for the rest of your nodes

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/066a7a52-c011-44c3-b630-953c82ab4818%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/066a7a52-c011-44c3-b630-953c82ab4818%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1UpZHXYuTWPtJOjXFKE7wPfwQe4puaT30yM32KfDHESw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

5

On Friday, December 19, 2014 1:01:53 PM UTC-8, Nikolas Everett wrote:

I believe so.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/cf15c260-866d-431b-9cf2-093ab8881f6f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.