I've restarted the cluster a couple times since and not seen what I saw
Been reading more of the documentation, am going to set the "min-max
master" to 3 which is suggested for a 5 node cluster.
Currently speculating, although I thought I've been very careful to start
the master significantly before any other node, something may have happened
the time that causes a persistently red status.
Questions related to this general topic (restarting a cluster)
Q - Once a cluster has started up with a different node as the master, is
there persistence in continuing to assign that role to that node or is it
completely arbitrary on every startup (ie what are the attributes an
election is based on)?
Q - If a cluster has started up with the wrong nodes with the Master role,
is it possible or advisable to try to modify their roles while the cluster
is running or is it advisable to shutdown the cluster, re-configure and
start up again?
On Tuesday, February 4, 2014 2:27:30 AM UTC-8, Boaz Leskes wrote:
A couple of points:
- If you bring down a whole cluster and start it back up, it may be that
during the start process the cluster is red. The reason is that until all
nods have rejoined some data may not be (yet) available for searching. This
should be resolve as soon as all the nodes are back (potentially earlier
depending on your replication settings)
- Though not recommended - kill -9 should not result in data loss. If
so it's a bug and should be reported.
On Monday, February 3, 2014 11:15:24 PM UTC+1, Tony Su wrote:
Thx for the input.
Nope, ES is being shutdown "normally" usually by simply stopping the
configured ES service, and only after it fully completes executing a
On Monday, February 3, 2014 2:09:08 PM UTC-8, InquiringMind wrote:
You're not doing a kill -9 during shutdown, I hope. If so, that would
result in a large window of opportunity for index corruption.
Just something to check for...
We always do a normal kill to the pid within the pid file to shut down
an ES instance before shutting down the machine itself, or before upgrading
the software.And we have never seen any issues with the cluster coming back
up in the same (usable, usually yellow or green) state that it was before
On two occasions we have had machines power off due to thermal overload
in the server room. This is a drastic event that is usually as dangerous
(to disk data integrity) as a kill -9, but in these cases there wasn't any
load on the machine and we experienced no data loss nor did we see the
cluster as anything but green once the machine came back up and the node
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to email@example.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/89b4183e-5cf6-4c04-ab21-b4ab427fc313%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.