Dumb Question on bouncing nodes


(Pierce Wetter) #1

Q: Is using the automatic node naming where every time I bounce a node I
get a new node name interfering with restarting from the local gateway?

Q Background:

So previously, I asked about bouncing nodes, because it seems that
whenever I bounce a node, it has to regenerate the node from the other
nodes in the cluster.

Someone told me I needed to lock shard allocation, then bounce, because
the problem was that the cluster was reallocating the shards off of the
node, so then the node wouldn't just
come back up from the local gateway.

I've tried that, and it doesn't seem to work. What happens is when the
instance shutdown, all of its shards go into "unallocated". Then when I
startup the instance, the node comes up with zero shards until I unlock
shard allocation. So the locking didn't seem to help.

I'm wondering if the problem is because all of my nodes are named
automatically using the marvel characters list, so essentially the node
name changes each time. So the lock/unlock doesn't help, because the
cluster is expecting a node with a specific name and when it doesn't find
it, it initializes the node with no shards.

Pierce

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5a6401b1-d6a9-4aad-807f-01f4c47a63d7%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Ivan Brusic) #2

Unless I misread the post, your scenario seems to be the normal behavior.
If you restart a node with allocation disabled, it will have no shards.
Re-enabling allocation will force the shards on the node to be active. This
behavior is at least what I have experienced. Perhaps there should be a
setting to ignore cluster allocation settings for new nodes if it contains
local shards that the cluster deems as unassigned.

There was a recent commit that dealt with favoring local shards upon
startup. https://github.com/elasticsearch/elasticsearch/issues/4237

Haven't upgraded to 0.90.8+ yet, but perhaps it addresses this issue.

Cheers,

Ivan

On Tue, Dec 31, 2013 at 11:25 AM, Pierce Wetter obastard@gmail.com wrote:

Q: Is using the automatic node naming where every time I bounce a node I
get a new node name interfering with restarting from the local gateway?

Q Background:

So previously, I asked about bouncing nodes, because it seems that
whenever I bounce a node, it has to regenerate the node from the other
nodes in the cluster.

Someone told me I needed to lock shard allocation, then bounce, because
the problem was that the cluster was reallocating the shards off of the
node, so then the node wouldn't just
come back up from the local gateway.

I've tried that, and it doesn't seem to work. What happens is when the
instance shutdown, all of its shards go into "unallocated". Then when I
startup the instance, the node comes up with zero shards until I unlock
shard allocation. So the locking didn't seem to help.

I'm wondering if the problem is because all of my nodes are named
automatically using the marvel characters list, so essentially the node
name changes each time. So the lock/unlock doesn't help, because the
cluster is expecting a node with a specific name and when it doesn't find
it, it initializes the node with no shards.

Pierce

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5a6401b1-d6a9-4aad-807f-01f4c47a63d7%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDN%3DMsQtK_n4pX0bD-%2B-Jv27ydcQKdyZ1erp2eNxAMLhQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Pierce Wetter) #3

Reading the description of that issue, I don't think they're the same.

Looking at the code... Yeah, that's only about moving primaries around.

So is there a way to restart a node without it going to zero shards and
then back again?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/25b969c7-ac3a-414a-b4e4-070b95b4857c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Ivan Brusic) #4

I do not think there is a way, but perhaps someone else can correct me. One
potential way is to instead of disabling allocation, to set a timeout value
high enough that the node can fully restart before the cluster drops the
node. However, I am not sure if a node that is being cleanly shutdown will
send an explicit message to the cluster. In that case, you would need a
more abrupt method of stopping the node.

Cheers,

Ivan

On Tue, Dec 31, 2013 at 12:27 PM, Pierce Wetter obastard@gmail.com wrote:

Reading the description of that issue, I don't think they're the same.

Looking at the code... Yeah, that's only about moving primaries around.

So is there a way to restart a node without it going to zero shards and
then back again?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/25b969c7-ac3a-414a-b4e4-070b95b4857c%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQB5S8FUcHC-6DdPgBcFSJtk0GRjsPWCzZH2RDFdMSS5-w%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #5