# Shrink using ILM **URL:** \ **Category:** Elasticsearch\ **Created:** [April 24, 2020, 2:45pm UTC](https://discuss.elastic.co/t/shrink-using-ilm/229677 "2020-04-24T14:45:21Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Phandora](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/phandora/32/46108_2.png) [@Phandora](https://discuss.elastic.co/u/Phandora)\ **Post date:** [April 24, 2020, 2:45pm UTC](https://discuss.elastic.co/t/shrink-using-ilm/229677/1 "2020-04-24T14:45:21Z")
Hi, We are designing our cluster taking into consideration some Elasticsearch best practices, focusing on keeping the shard size as close to our desired one (currently 25gb) as possible. We decided to create **ILM policies** in order to achieve that, by rolling the indices using `max_size` and `max_age` to simplify the index management. However, age rotation may lead to smaller indices. The **shrink** action would be a good choice in this case, but handling it with ILM seems to have certain limitations since the only setting available is `number_of_shards`. It would be more convenient to shrink the indices taking into account the index primary shards and its size since we can not know in advance which number of shards will be accurate. Is there any feature that might allow us to achieve this? Can it be approached differently? I would appreciate any help. Best regards. ---
**Author:** ![rugenl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rugenl/32/12887_2.png) [@rugenl](https://discuss.elastic.co/u/rugenl)\ **Post date:** [April 24, 2020, 3:18pm UTC](https://discuss.elastic.co/t/shrink-using-ilm/229677/2 "2020-04-24T15:18:39Z")
It's pretty much a trade off between retention granularity and shard size. Say I want to retain data 6 months, if I set rollover age to 15 days, some data will get the extra 15 days, some won't, assuming the rollover occurs because of age. It's still better than daily indices. Since we are multi-tenant, IMHO, tuning for cluster performance instead of individual index performance, I can almost always use 1 shard (and 1 replica). Since many indices are ingesting, the ingest load balances over available nodes anyway, each node with a few active indices. Since these tend to rollover at 50G in a day or two, searching is typically spread over the past week, so still multiple shards are involved in searches, and other users search other indices anyway, so search load is distributed. ---
**Author:** ![Phandora](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/phandora/32/46108_2.png) [@Phandora](https://discuss.elastic.co/u/Phandora)\ **Post date:** [April 24, 2020, 4:45pm UTC](https://discuss.elastic.co/t/shrink-using-ilm/229677/3 "2020-04-24T16:45:04Z")
In our case, more than 95% of the documents are ingested in the same index, therefore setting only 1 primary shard will undercut the elasticsearch ingest performance. Since the workload is variable (it always is), index management is a must to keep a good cluster performance. However, shrinking the index using ILM does not allow us to adjust the proper number of shards. For instance: Assuming **25gb** as our desired shard size and a `rollover policy with max_size=150gb` and `shrink with number_of_shards=3`. > Index\_1 --\> primary\_shards: 6; pri.store.size=150gb --\> After shrink : **too few shards** > Index\_2 --\> primary\_shards: 6; pri.store.size=70gb --\> After shrink : **desired shards** > Index\_3 --\> primary\_shards: 6; pri.store.size=20gb --\> After shrink : **too many shards** ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 22, 2020, 4:45pm UTC](https://discuss.elastic.co/t/shrink-using-ilm/229677/4 "2020-05-22T16:45:06Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.