They both are master eligible and replicas of each other. I need to rebuild M1. But when I do that and reinstall everything on this box. M1 doesn't join back M2. I am assuming that M2 is eligible but not becoming master and may have to do with "voting rights".
How do I make it work so after a hardware and OS upgrade M1 will join M2 back and become the master and copy the data
If you only have 2 master eligible nodes I believe you can not just rebuild one server as you then lose half the master eligible nodes which means a master can no longer be elected. Before you rebuild, make sure you take a snapshot and then remove the master node to be rebuilt.
Set master-eligible false on M1 node and restart it so it's data node only, then after cluster is stable, stop & rebuild M1. If need to rebuild later, swap master roles (M1.true, rejoin, M2.false, rejoin, stop & rebuild).
Add small voting-only node somewhere, then rebuild M1 any time.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.