# Internal monitoring and log indices have "live forever" ILM policies

**URL:** <https://discuss.elastic.co/t/internal-monitoring-and-log-indices-have-live-forever-ilm-policies/330072>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-monitoring, ilm-index-lifecycle-management\
**Created:** [April 15, 2023, 2:23pm UTC](https://discuss.elastic.co/t/internal-monitoring-and-log-indices-have-live-forever-ilm-policies/330072 "2023-04-15T14:23:41Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![ppine7](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppine7/32/111560_2.png) [@ppine7](https://discuss.elastic.co/u/ppine7)\
**Post date:** [April 15, 2023, 2:23pm UTC](https://discuss.elastic.co/t/internal-monitoring-and-log-indices-have-live-forever-ilm-policies/330072/1 "2023-04-15T14:23:41Z")

</div>

Hi!  
I was investigating an issue with too much retained data in our ES cloud cluster and realized that a lot of it comes from the internal monitoring and log indices. Specifically, the following indices with corresponding ILM policies:

- .ds-metricbeat-\* indices --\> "metricbeat" ILM policy

- .ds-elastic-cloud-logs-\* --\> "elastic-cloud-logs" ILM policy

Here is one of the policies - the other one is the same:

```auto
{
  "elastic-cloud-logs": {
    "version": 1,
    "modified_date": "2022-11-08T16:39:52.102Z",
    "policy": {
      "phases": {
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_primary_shard_size": "50gb",
              "max_age": "30d"
            }
          }
        }
      }
    },
    "in_use_by": {
      "indices": [
        ".ds-elastic-cloud-logs-8-2023.02.08-000005",
        ".ds-elastic-cloud-logs-8-2023.03.10-000006",
        ".ds-elastic-cloud-logs-8-2023.04.09-000007",
        ".ds-elastic-cloud-logs-8-2023.01.09-000004",
        ".ds-elastic-cloud-logs-8-2023.01.07-000003"
      ],
      "data_streams": [
        "elastic-cloud-logs-8"
      ],
      "composable_templates": [
        "elastic-cloud-logs-8"
      ]
    }
  }
}

```

I know I can change them manually (or add to our automation) however I like, but:  
a few questions:

1. is this OK/advisable to change internal ES ILM policies?
2. and if I do that - would they be, potentially ,updated when the managed ES cluster is updated? (for example, reverted back tot he default version)?
3. and related to above - DO ES cluster upgrade automatically for minor versions? or is it a manually "kicked off" process only?

Thank you!  
Marina

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [April 17, 2023, 4:16pm UTC](https://discuss.elastic.co/t/internal-monitoring-and-log-indices-have-live-forever-ilm-policies/330072/2 "2023-04-17T16:16:23Z")

</div>

Hi @ppine7

> [@ppine7](#):
>
> - is this OK/advisable to change internal ES ILM policies?
> 
> > Yes it is OK to do that...
> 
> - and if I do that - would they be, potentially ,updated when the managed ES cluster is updated? (for example, reverted back tot he default version)?
> 
> > Hmmm I do not think so... I don't have a way to test at the moment
> 
> - and related to above - DO ES cluster upgrade automatically for minor versions? or is it a manually "kicked off" process only?
> 
> > All updates are user initiated, no automatic updates of Clusters

---

<div class="post-metadata">

**Author:** ![ppine7](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppine7/32/111560_2.png) [@ppine7](https://discuss.elastic.co/u/ppine7)\
**Post date:** [April 17, 2023, 4:32pm UTC](https://discuss.elastic.co/t/internal-monitoring-and-log-indices-have-live-forever-ilm-policies/330072/3 "2023-04-17T16:32:21Z")

</div>

Great, thank you, @stephenb !

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 15, 2023, 4:33pm UTC](https://discuss.elastic.co/t/internal-monitoring-and-log-indices-have-live-forever-ilm-policies/330072/4 "2023-05-15T16:33:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.