The following has bearing on understanding what happens to the data store
when Replicas are increased or decreased.
Reading about re-sizing the cluster, I've read
Changing the number of shards requires re-indexing, so it's highly
recommended to allocate a number of shards consistent with the max number
of expected nodes in the cluster from the beginning.
Replicas can be added or removed at any time without an unusual load.
So, it begs the question, I would expect that adding replicas creates new
copies.
But, what happens if the number of replicas is decreased?
Is there a difference changing replica number by runtime command compared
to starting up a node (elasticsearch.yml), particularly if replica numbers
are decreased?
I assume that the replica number with the highest number is removed first?
And what about the data, is it automatically purged or left "orphaned" in
data storage?
And if orphaned in data storage, is there a recommended way to manually
remove the replica data no longer used?
I would also expect that after decreasing the number of replicas, if the
number is increased again that whole new replicas would be created and
allocated, none of the old replicas might be checked for integrity and
possibly re-used (same procedure as recovery).
When you decrease the replicas dynamically, the data is deleted from
wherever nodes they reside in. Then when you increase it back dynamically,
ES will then make new copies of the shards on-the-fly. Also when you set
index.number_of_replicas in the YML file, that only applies to new indexes
created after start up. If you already had an existing index prior to that
change in the YML file, no change will be made to the replica count of that
prior existing index.
Thank you for what appears to be a complete explanation describing
all scenarios, both runtime changes and startup (based on
elasticsearch.yml).
Enlightening that there are significant differences.
Tony
On Tuesday, February 18, 2014 7:48:16 AM UTC-8, Binh Ly wrote:
When you decrease the replicas dynamically, the data is deleted from
wherever nodes they reside in. Then when you increase it back dynamically,
ES will then make new copies of the shards on-the-fly. Also when you set
index.number_of_replicas in the YML file, that only applies to new indexes
created after start up. If you already had an existing index prior to that
change in the YML file, no change will be made to the replica count of that
prior existing index.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.