# Can ES correctly re-balance shards when all master nodes down? **URL:** \ **Category:** Elasticsearch\ **Created:** [January 6, 2016, 3:59am UTC](https://discuss.elastic.co/t/can-es-correctly-re-balance-shards-when-all-master-nodes-down/38461 "2016-01-06T03:59:10Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Youxu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youxu/32/117375_2.png) [@Youxu](https://discuss.elastic.co/u/Youxu)\ **Post date:** [January 6, 2016, 3:59am UTC](https://discuss.elastic.co/t/can-es-correctly-re-balance-shards-when-all-master-nodes-down/38461/1 "2016-01-06T03:59:10Z")
Our cluster has 3 dedicated master nodes, 3 dedicated client nodes and several data nodes. I know ES query still works even all master nodes down. But what will happen when some of data nodes down? Can ES still re-balance the shards from those down data nodes? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 6, 2016, 4:43am UTC](https://discuss.elastic.co/t/can-es-correctly-re-balance-shards-when-all-master-nodes-down/38461/2 "2016-01-06T04:43:15Z")
> [@Youxu](#): > > I know ES query still works even all master nodes down. No, it won't. If _all_ master eligible nodes are unavailable then the other nodes will stop serving requests as they have nothing maintaining the cluster. > [@Youxu](#): > > But what will happen when some of data nodes down? Can ES still re-balance the shards from those down data nodes? If only data nodes go down and there is adequate master nodes available, then yes ES will rebalance. ---
**Author:** ![sandesh](https://avatars.discourse-cdn.com/v4/letter/s/d6d6ee/32.png) [@sandesh](https://discuss.elastic.co/u/sandesh)\ **Post date:** [January 6, 2016, 6:23am UTC](https://discuss.elastic.co/t/can-es-correctly-re-balance-shards-when-all-master-nodes-down/38461/3 "2016-01-06T06:23:39Z")
HI, ES node goes down means it shows unresponsiveness or noes is shut down. ---
**Author:** ![Youxu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youxu/32/117375_2.png) [@Youxu](https://discuss.elastic.co/u/Youxu)\ **Post date:** [January 7, 2016, 2:51am UTC](https://discuss.elastic.co/t/can-es-correctly-re-balance-shards-when-all-master-nodes-down/38461/4 "2016-01-07T02:51:47Z")
Thanks your reply. Per our testing, the query DID work after we shutdown all dedicated master nodes. Our query endpoints is on dedicated client nodes. As for re-balancing, in my understanding, re-balancing need master nodes involvement, so if all mater nodes down, ES could not do shard re-balance when any data node down. But the problem is, if all master nodes down, the head plugin as well as \_cat API won't work, I am not able to verify whether the shards are re-balanced or NOT. Do you know how to verify? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 7, 2016, 3:50am UTC](https://discuss.elastic.co/t/can-es-correctly-re-balance-shards-when-all-master-nodes-down/38461/5 "2016-01-07T03:50:08Z")
> [@Youxu](#): > > Per our testing, the query DID work after we shutdown all dedicated master nodes Then you must have other nodes that can become masters. ---
**Author:** ![Youxu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youxu/32/117375_2.png) [@Youxu](https://discuss.elastic.co/u/Youxu)\ **Post date:** [January 8, 2016, 2:35am UTC](https://discuss.elastic.co/t/can-es-correctly-re-balance-shards-when-all-master-nodes-down/38461/6 "2016-01-08T02:35:45Z")
NO. I am 100% sure that all my master eligible nodes were down. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 8, 2016, 3:50am UTC](https://discuss.elastic.co/t/can-es-correctly-re-balance-shards-when-all-master-nodes-down/38461/7 "2016-01-08T03:50:07Z")
Can you post the output from `_cat/nodes`? ---
**Author:** ![Youxu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youxu/32/117375_2.png) [@Youxu](https://discuss.elastic.co/u/Youxu)\ **Post date:** [January 8, 2016, 5:59am UTC](https://discuss.elastic.co/t/can-es-correctly-re-balance-shards-when-all-master-nodes-down/38461/8 "2016-01-08T05:59:51Z")
the problem is, when all master nodes down, the \_cat API does not work, although the query still works. BTW: I sent all request to client nodes. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 11:26pm UTC](https://discuss.elastic.co/t/can-es-correctly-re-balance-shards-when-all-master-nodes-down/38461/9 "2017-07-05T23:26:11Z")