Hello,
I have a test cluster that has 2 nodes. I want to change the configuration, to add 3 master nodes, so my 2 initial nodes serve just as data nodes.
So this is what I tried:
I start my new 3 masters, and they become one cluster.
Then I disable shard allocation on my 2 node cluster, change the settings to specify they are to be just data nodes. Restart the 2 nodes. They join the 3 node cluster, which is now 5 node (3 master 2 nodes). I enable shard allocation. But my cluster remains red, with about half unassigned shards. When I list shards, the only "STARTED" are on one data node. The other data node doesn't have any shards.
Why are shards not pushed to the one node? My cluster health:
{
"cluster_name" : "tlsaasprodqa",
"status" : "red",
"timed_out" : false,
"number_of_nodes" : 5,
"number_of_data_nodes" : 2,
"active_primary_shards" : 1354,
"active_shards" : 1354,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 1356,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0
}
my cluster settings:
"cluster" : {
"routing" : {
"allocation" : {
"enable" : "all",
"node_initial_primaries_recoveries" : "50",
"cluster_concurrent_rebalance" : "0",
"node_concurrent_recoveries" : "20"
}
}
}
},
"transient" : {
"cluster" : {
"routing" : {
"allocation" : {
"enable" : "all",
"node_concurrent_recoveries" : "20"
}
}
}
}
If I kill ES on 3 master nodes and set one of the data nodes to be data/master, these shards will be assigned.
What am I doing wrong? Is this the proper way to add 3 master nodes?
All nodes have the same version, 1.7.1.
Thanks