Scaling down ElasticSearch: what happens with closed indices?

[caarlos0 (Carlos Alexandro Becker)]

To scale down ElasticSearch, I usually disable allocation on a node, wait for shards to relocate, and then remove the node, e.g.:

curl -X PUT "localhost:9200/_cluster/settings" -d'{
  "transient" : {
    "cluster.routing.allocation.exclude._name" : "node-4"
  }
}'

As I understand, open indices' shards will be relocated to other nodes... but what happens to closed indices?

A closed index is, per my understanding, ignored on snapshots... so it makes me think if it will be ignored when setting an allocation exclusion as well.

After excluding allocation to a node, I went to check its disk usage... and it seems like it still has a lot of things there:

$ du -hs /usr/share/elasticsearch/nodes/0/
131G	/usr/share/elasticsearch/nodes/0/

So, my questions are:

• What happens to closed indices when I exclude a node from allocation?
• How safe it is to do this if I don't want to loose those closed indices (for whatever reason)?
• Is there a way to tell ES to also rebalance closed indices in case it doesn't already do it?

[warkolm (Mark Walkom)]

It depends on what version you are on. More recent versions have replicas for closed indices, but I am not 100% sure when that landed.

I am not sure how allocation works for them though sorry.

[caarlos0 (Carlos Alexandro Becker)]

Thanks, I'm on 5.6.16... didn't find anything on the docs

[warkolm (Mark Walkom)]

Ahh ok, yeah that's not going to apply then sorry, that's definitely a later 7.X feature.

I will highlight that 5.X reached EOL some time ago, and you should look at upgrading as a matter of urgency

[caarlos0 (Carlos Alexandro Becker)]

Yes, need to upgrade ASAP, unfortunately also need to redesign a lot of things to do it hehe

[Christian_Dahlqvist]

If you are on Elasticsearch 5.x you will need to open them in order to relocate the shards.