# Curator should be able to shrink indices by size

**URL:** <https://discuss.elastic.co/t/curator-should-be-able-to-shrink-indices-by-size/176740>\
**Category:** Elasticsearch\
**Tags:** curator\
**Created:** [April 13, 2019, 4:28pm UTC](https://discuss.elastic.co/t/curator-should-be-able-to-shrink-indices-by-size/176740 "2019-04-13T16:28:20Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![DewinGoh](https://avatars.discourse-cdn.com/v4/letter/d/a88e4f/32.png) [@DewinGoh](https://discuss.elastic.co/u/DewinGoh)\
**Post date:** [April 13, 2019, 4:28pm UTC](https://discuss.elastic.co/t/curator-should-be-able-to-shrink-indices-by-size/176740/1 "2019-04-13T16:28:20Z")

</div>

Hi all, I've been experimenting with Elasticsearch and curator for a bit, but am stuck on figuring out how to shrink indices by their sizes.

My current problem is as such: I have been handed a cluster with over 2000 shards per node (clearly, some poor cluster management here!), and am trying to reduce the shard count by shrinking the number of shards back to indices. The issue is that the shards are all highly imbalanced in size, ranging from a few KBs to 100+ GBs. I have created templates for the incoming indices already, but there are still tens of thousands of shards to handle downstream. What I'm planning to do is to shrink shards smaller than 25GB in size back to a single shard. From what I have explored so far, curator seems to have no option to do curate by index size - the closest I see is the "space" filter. When I tried it, curator kept terminating the moment the next index was \>25GB, due to the cumulative sum implementation of the disk\_space function.

What I do suggest is either 1) curator should be able to have a filtertype by index size, or 2) in the implementation of disk\_space, allow sorting by index size

At the same time, I'd like to also verify with the community if the way I'm approaching this issue is appropriate. The cluster details are as such: Elasticsearch v6.3.2, Curator 5.5.4, 3 Master 3 Client 9 Data Nodes, all run on Kubernetes.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [April 13, 2019, 7:25pm UTC](https://discuss.elastic.co/t/curator-should-be-able-to-shrink-indices-by-size/176740/2 "2019-04-13T19:25:50Z")

</div>

You mean, like [the disk space filter](https://www.elastic.co/guide/en/elasticsearch/client/curator/5.6/filtertype_space.html)? You need to do more filtering besides space. Like a combination of pattern filters and age filters to identify indices _within_ a certain time frame to identify indices of a certain kind _and_ size.

It's unlikely that Curator will be modified to do the kind of disk-consumption-based approach you seek, as it's an edge case. You're probably better off re-indexing multiple small indices into larger ones, rather than try to winnow them out by size and then go from there.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 11, 2019, 7:25pm UTC](https://discuss.elastic.co/t/curator-should-be-able-to-shrink-indices-by-size/176740/3 "2019-05-11T19:25:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.