# Reduce (shrink) primary shards on older data indices of a data stream

**URL:** https://discuss.elastic.co/t/reduce-shrink-primary-shards-on-older-data-indices-of-a-data-stream/293963
**Category:** Elasticsearch
**Tags:** ilm-index-lifecycle-management, reindex, datastreams
**Created:** [January 11, 2022, 2:27am UTC](https://discuss.elastic.co/t/reduce-shrink-primary-shards-on-older-data-indices-of-a-data-stream/293963 "2022-01-11T02:27:57Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![adriaan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriaan/32/100196_2.png) [@adriaan](https://discuss.elastic.co/u/adriaan)
#### Post date: [January 11, 2022, 2:27am UTC](https://discuss.elastic.co/t/reduce-shrink-primary-shards-on-older-data-indices-of-a-data-stream/293963/1 "2022-01-11T02:27:57Z")

</div>

I have a data stream with page views of multiple websites of customers. I allow customers to delete their data (this is important). When they do I remove all the documents with a certain hostname.

At the same time, I want to keep my search of recent data fast. What works best is having 12 primary shards of recent data. Older data is fine with 3 shards (both on a 3 node cluster).

The most recent data (not older than 1 day) will also be updated with a few extra metrics we collect at the end of the page views (like time on page).

A few options (and their concerns):

- The [shrink API](https://www.elastic.co/guide/en/elasticsearch/reference/current/ilm-shrink.html) but it is creating read-only indices (I need to be able to delete old but specific data)
- Make the data stream writable once every month and run the deletions then (is that possible?)
- Reindex the whole data stream every month to remove the older documents (how to keep writes and updates in the last month working well?)

What would be a way to make the recent data fast in a data stream that sometimes gets updates (in very old data).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 8, 2022, 2:28am UTC](https://discuss.elastic.co/t/reduce-shrink-primary-shards-on-older-data-indices-of-a-data-stream/293963/2 "2022-02-08T02:28:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
