# Apply ILM on rollup or transform index

**URL:** <https://discuss.elastic.co/t/apply-ilm-on-rollup-or-transform-index/294039>\
**Category:** Elasticsearch\
**Tags:** ilm-index-lifecycle-management\
**Created:** [January 11, 2022, 1:45pm UTC](https://discuss.elastic.co/t/apply-ilm-on-rollup-or-transform-index/294039 "2022-01-11T13:45:43Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Achraf\_MENSI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/achraf_mensi/32/100145_2.png) [@Achraf\_MENSI](https://discuss.elastic.co/u/Achraf_MENSI)\
**Post date:** [January 11, 2022, 1:45pm UTC](https://discuss.elastic.co/t/apply-ilm-on-rollup-or-transform-index/294039/1 "2022-01-11T13:45:43Z")

</div>

Hello, does anyone have successfully applied ILM on a transform or rollup index? or it's possible only with normal index? thanks

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [January 11, 2022, 10:30pm UTC](https://discuss.elastic.co/t/apply-ilm-on-rollup-or-transform-index/294039/2 "2022-01-11T22:30:52Z")

</div>

Welcome to our community! 😃

Are you having problems implementing this?

---

<div class="post-metadata">

**Author:** ![Achraf\_MENSI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/achraf_mensi/32/100145_2.png) [@Achraf\_MENSI](https://discuss.elastic.co/u/Achraf_MENSI)\
**Post date:** [January 13, 2022, 11:30am UTC](https://discuss.elastic.co/t/apply-ilm-on-rollup-or-transform-index/294039/3 "2022-01-13T11:30:02Z")

</div>

yes i want to apply ILM on an sumuriezed part of my data for 2 months ( transfrom or rollup ) without duplicate data that is principale index in hot phase.

---

<div class="post-metadata">

**Author:** ![rfox](https://avatars.discourse-cdn.com/v4/letter/r/b4bc9f/32.png) [@rfox](https://discuss.elastic.co/u/rfox)\
**Post date:** [January 24, 2022, 9:36pm UTC](https://discuss.elastic.co/t/apply-ilm-on-rollup-or-transform-index/294039/4 "2022-01-24T21:36:41Z")

</div>

> [@Achraf\_MENSI](#):
>
> LM on an sumuriezed part of

I too am interested in how to accomplish this. I want to age out old rollup data through the ILM stages.

---

<div class="post-metadata">

**Author:** ![Achraf\_MENSI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/achraf_mensi/32/100145_2.png) [@Achraf\_MENSI](https://discuss.elastic.co/u/Achraf_MENSI)\
**Post date:** [January 27, 2022, 2:23pm UTC](https://discuss.elastic.co/t/apply-ilm-on-rollup-or-transform-index/294039/6 "2022-01-27T14:23:20Z")

</div>

i think it's not possible so i used transform api instead of rollup..

---

<div class="post-metadata">

**Author:** ![rfox](https://avatars.discourse-cdn.com/v4/letter/r/b4bc9f/32.png) [@rfox](https://discuss.elastic.co/u/rfox)\
**Post date:** [January 27, 2022, 5:15pm UTC](https://discuss.elastic.co/t/apply-ilm-on-rollup-or-transform-index/294039/7 "2022-01-27T17:15:46Z")

</div>

Ok. How did that work? Doesn't transform have the same problem that it is one named index and not a index pattern that is used to move the data through ILM?

---

<div class="post-metadata">

**Author:** ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)\
**Post date:** [January 27, 2022, 7:55pm UTC](https://discuss.elastic.co/t/apply-ilm-on-rollup-or-transform-index/294039/8 "2022-01-27T19:55:15Z")

</div>

ILM and Transform

For using ILM on the destination index, you have to create the destination upfront _before_ starting the transform. Note: It is not sufficient to create an index template, but you have to create the index. It is possible to let transform write to an index alias, write aliases are supported, too. Transform will only create the destination index for you, if either an index or alias does not exist at the time you start the transform. You can use the preview API to see what transform suggests as index mappings.

Problem explained:

Transform updates documents in the destination, this can lead to side effects together with the usage of ILM. Transform will only write to the current destination, in case of a rollover this might create duplicates, because transform can not delete documents in the previous concrete index.

Possible Solution:

If you have time-based indices and your transform contains a `group_by` based on `date_histogram` you can attach an ingest pipeline to `dest` and configure a [date index name processor](https://www.elastic.co/guide/en/elasticsearch/reference/current/date-index-name-processor.html) with the timestamp field from the group by as input. This won't by design create duplicates. The idea is basically that by using the index name processor that you create daily/weekly/monthly indices. This isn't based on size, but maybe that's ok for you.

---

<div class="post-metadata">

**Author:** ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)\
**Post date:** [January 28, 2022, 6:50am UTC](https://discuss.elastic.co/t/apply-ilm-on-rollup-or-transform-index/294039/9 "2022-01-28T06:50:59Z")

</div>

Addition:

If you use a recent version of the stack `align_checkpoints` which is part of `settings` - default `true` - allows you to only write complete buckets.

If _and only if_ the time field used in `sync` and the timefield used for the `date_histogram` in `group_by` are the same the problem of duplicates in the destination due to index rollover is technically not possible. For this special case just create the destination youself and configure ILM for it.

If your `sync` field and the `date_histogram` field are not the same, this won't work reliable and I suggest the index name trick explained above.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 25, 2022, 6:51am UTC](https://discuss.elastic.co/t/apply-ilm-on-rollup-or-transform-index/294039/10 "2022-02-25T06:51:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
