# To big index for ML job - how to add ILM?

**URL:** <https://discuss.elastic.co/t/to-big-index-for-ml-job-how-to-add-ilm/357611>\
**Category:** Elasticsearch\
**Created:** [April 17, 2024, 12:26pm UTC](https://discuss.elastic.co/t/to-big-index-for-ml-job-how-to-add-ilm/357611 "2024-04-17T12:26:20Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Wojciech\_Kwiecien](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wojciech_kwiecien/32/109829_2.png) [@Wojciech\_Kwiecien](https://discuss.elastic.co/u/Wojciech_Kwiecien)\
**Post date:** [April 17, 2024, 12:26pm UTC](https://discuss.elastic.co/t/to-big-index-for-ml-job-how-to-add-ilm/357611/1 "2024-04-17T12:26:21Z")

</div>

Hello Everyone,

I’ve encountered an issue while working with Machine Learning jobs in Elasticsearch, where the result indices aren't being properly managed post-rollover.  
I have to figured out something because index for this specific ML job was above 600 GB.  
I need any ILM and rollover strategy to manage this inidices.

**Problem Description:**  
I started by creating an ML job in Elasticsearch, specifying the `results_index_name` as a concrete index name ending with a sequence number (e.g., `custom-ml-anomalies-metrics-count-high_mean-v12-000001`). Everything worked fine until the index met the rollover conditions set in the ILM policy. The index was rolled over to a new index (e.g., `...-000002`), but the ML job continued writing to the old index, resulting in new data not being recorded in the newly created index.

**Steps Taken:**

1. I attempted to manually rollover the index using the Rollover API, but after that new `custom-ml-anomalies-metrics-count-high_mean-v12-000002` was not "assigned" to ML job.  
ML job stop adding a documents to `custom-ml-anomalies-metrics-count-high\_mean-v12-000001' and 'custom-ml-anomalies-metrics-count-high\_mean-v12-000002'
2. I created an alias for the result index with the `is_write_index` flag set to `true`, hoping this would solve the issue of updating the target index in the ML job.

**Configuration:**

- I set an alias for the result index (e.g., `ml-results-custom-metrics-v12`) as `results_index_name` in the ML job.
- I configured an ILM policy to use this alias as the `rollover_alias`.

**Questions:**  
Do you have any suggestions on how yo achieve my goal?

I would appreciate any comments and suggestions.

I followed by this [ROTATE ML INDEXES](https://discuss.elastic.co/t/rotate-ml-indexes/339286) but without luck.

---

<div class="post-metadata">

**Author:** ![GinkoLucas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ginkolucas/32/107055_2.png) [@GinkoLucas](https://discuss.elastic.co/u/GinkoLucas)\
**Post date:** [January 9, 2025, 11:11am UTC](https://discuss.elastic.co/t/to-big-index-for-ml-job-how-to-add-ilm/357611/2 "2025-01-09T11:11:56Z")

</div>

Hi,  
So you tried to use an alias if I understand correctly? Did it work?  
I don't know if this is possible, but have you tried using a datastream instead of bothering with an alias? To do this, the data leaving the ML Job must be time-stamped, but I don't know if this is the case.