# Active shards after closing indices **URL:** \ **Category:** Elasticsearch\ **Tags:** ilm-index-lifecycle-management\ **Created:** [May 8, 2020, 8:36am UTC](https://discuss.elastic.co/t/active-shards-after-closing-indices/231678 "2020-05-08T08:36:39Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![bartelastic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bartelastic/32/67939_2.png) [@bartelastic](https://discuss.elastic.co/u/bartelastic)\ **Post date:** [May 8, 2020, 8:36am UTC](https://discuss.elastic.co/t/active-shards-after-closing-indices/231678/1 "2020-05-08T08:36:40Z")
I see that after closing indices the shards are still reported as active (also see [Elasticsearch 7.x closed indices retain shards](https://discuss.elastic.co/t/elasticsearch-7-x-closed-indices-retain-shards/187969) ) and that was done to implement [replicated closed indices feature](https://github.com/elastic/elasticsearch/issues/33888) . Although I technically understand what to do now (increase max\_shards\_per\_node), I was still wondering if it was debated whether another stat could be used for the feature, like the frozen status, which also seems to be attached to the closed indices. Since Elasticsearch 7 the max\_shards\_per\_node setting is mandatory and I thought it was to be able to tune on the active shards. But now the frozen and closed indices report their shards as active, while the shards have a small footprint and thus do not compare with the resource usage of active shards in open indices. So the max\_shards\_per\_node doesn't really say something about the resource usage and has to be increased artificially to give room for the shards of closed and frozen indices. I am glad that we created monitoring for the percentage of allowed active shards 🙂 Just wondering whether any changes for this system were on the roadmap or what the deliberation on the pros and cons was. Kind regards Bart ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [May 8, 2020, 9:27am UTC](https://discuss.elastic.co/t/active-shards-after-closing-indices/231678/2 "2020-05-08T09:27:47Z")
There were indeed discussions on whether to include closed and frozen indices in the count used by `max_shards_per_node` and it was a deliberate decision to include them, although there are good arguments in both directions. Closed and frozen indices are not completely free (e.g. they must be tracked by the master and recovered on failures) and the default `max_shards_per_node` limit is a very coarse safety feature to protect the cluster from really bad cases of oversharding rather than a recommended target. Maintaining a large number of closed indices in your cluster is something of an antipattern. If you don't ever want to search them then it's better to offload them into a snapshot; if you do want to support occasional searches then it's better to freeze them. For frozen indices the limit makes a bit more sense since searching a large number of small shards is relatively inefficient, so it's recommended to do some extra work before freezing to get the most out of your system. For instance, you can consolidate the data into fewer larger indices via reindex, shrink the indices to fewer larger shards, and force-merge them to a single segment. ---
**Author:** ![bartelastic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bartelastic/32/67939_2.png) [@bartelastic](https://discuss.elastic.co/u/bartelastic)\ **Post date:** [May 8, 2020, 9:47am UTC](https://discuss.elastic.co/t/active-shards-after-closing-indices/231678/3 "2020-05-08T09:47:57Z")
Thanks for your reply! Ok, at the moment we are freezing them after a week and closing them a while later (time depending on whether it is our dev or prod environment). Maybe we shouldn't use the closed before deleting indices, as it adds little to the performance. And maybe do some extra work before freezing. For now it works well enough but data is growing, so it is best to keep that in mind. Apparently, I thought the `max_shards_per_node` as a less rough feature than it was intended. Thanks again for the clear explanation! ---
**Author:** ![bartelastic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bartelastic/32/67939_2.png) [@bartelastic](https://discuss.elastic.co/u/bartelastic)\ **Post date:** [May 8, 2020, 10:04am UTC](https://discuss.elastic.co/t/active-shards-after-closing-indices/231678/4 "2020-05-08T10:04:39Z")
One more question, would it be an idea to specifically mention this on [https://www.elastic.co/guide/en/elasticsearch/reference/current/allocation-total-shards.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/allocation-total-shards.html) [https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-close.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-close.html) and [https://www.elastic.co/guide/en/elasticsearch/reference/current/freeze-index-api.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/freeze-index-api.html) ? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [May 8, 2020, 10:32am UTC](https://discuss.elastic.co/t/active-shards-after-closing-indices/231678/5 "2020-05-08T10:32:01Z")
I think there is indeed a bit of scope to improve the docs. - I think [https://www.elastic.co/guide/en/elasticsearch/reference/7.6/allocation-total-shards.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/allocation-total-shards.html) isn't relevant, that's to do with allocation rather than oversharding. - I was going to suggest improving [https://www.elastic.co/guide/en/elasticsearch/reference/7.6/misc-cluster.html#cluster-shard-limit](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/misc-cluster.html#cluster-shard-limit) but I see that this actually already says that closed indices are _not_ counted here, sorry. Should have checked that, I remember the discussions but clearly forgot the exact outcome 🙂 - [https://www.elastic.co/guide/en/elasticsearch/reference/7.6/best\_practices.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/best_practices.html) does mention force-merging but could also reasonably mention shrinking, and could also point out that although frozen indices are lighter than non-frozen ones they can still be a source of oversharding (ref. [https://www.elastic.co/guide/en/elasticsearch/reference/7.6/avoid-oversharding.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/avoid-oversharding.html)) and in particular that [https://www.elastic.co/guide/en/elasticsearch/reference/6.8/misc-cluster.html#cluster-shard-limit](https://www.elastic.co/guide/en/elasticsearch/reference/6.8/misc-cluster.html#cluster-shard-limit) applies to them too. - [https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-close.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-close.html) could IMO bear a suggestion to consider snapshots or frozen indices instead of long-term closed indices. All that would be subject to some internal discussion too, we have to be careful that the reference docs don't miss any subtleties, but a PR to do some or all of those changes would be welcome. ---
**Author:** ![bartelastic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bartelastic/32/67939_2.png) [@bartelastic](https://discuss.elastic.co/u/bartelastic)\ **Post date:** [May 8, 2020, 10:44am UTC](https://discuss.elastic.co/t/active-shards-after-closing-indices/231678/6 "2020-05-08T10:44:26Z")
I'll work on that ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 5, 2020, 10:44am UTC](https://discuss.elastic.co/t/active-shards-after-closing-indices/231678/7 "2020-06-05T10:44:34Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.