# Sudden rollover of apm indices

**URL:** <https://discuss.elastic.co/t/sudden-rollover-of-apm-indices/355473>\
**Category:** Elasticsearch\
**Tags:** ilm-index-lifecycle-management\
**Created:** [March 15, 2024, 11:07am UTC](https://discuss.elastic.co/t/sudden-rollover-of-apm-indices/355473 "2024-03-15T11:07:00Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Harsh\_Jain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harsh_jain/32/132648_2.png) [@Harsh\_Jain](https://discuss.elastic.co/u/Harsh_Jain)\
**Post date:** [March 15, 2024, 11:07am UTC](https://discuss.elastic.co/t/sudden-rollover-of-apm-indices/355473/1 "2024-03-15T11:07:00Z")

</div>

Hi,  
I am new to elastic apm and have been facing issue with ilm policies.

Following tasks are done:

1. Edited metrics-apm.app\_metrics-default\_policy and saved as custom-metrics-apm.app\_metrics-default\_policy
2. Updated ilm policy in metrics-apm.app@custom component template. Applied and saved it.
3. Then chose the option to wait for next rollover and apply while saving the template.

Next day, suddenly all the metrics-apm-default indices are rolled-over, even though the condition of 30 days was not met.

To rule out the interference of fleet, I created a new ilm policy new-metrics-apm.app\_metrics-default\_policy, with 90d or 50GB max primary shard condition in hot phase; and 180d deletion phase age.  
Then I have updated it in metrics-apm.app@custom component template. Applied and saved it.  
This time while saving I chose Rollover now and apply policy.  
All the indices got rolled-over immediately, as expected.

But after ~5 min, all the metrics-apm-default indices got rolled-over automatically.

Please help me understand this abrupt rollover behaviour.

Elasticsearch version: 8.8.0  
APM server version: 8.8.1  
No fleet server was used in the installation.

---

<div class="post-metadata">

**Author:** ![Harsh\_Jain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harsh_jain/32/132648_2.png) [@Harsh\_Jain](https://discuss.elastic.co/u/Harsh_Jain)\
**Post date:** [March 26, 2024, 2:56pm UTC](https://discuss.elastic.co/t/sudden-rollover-of-apm-indices/355473/2 "2024-03-26T14:56:24Z")

</div>

Please reply. This issue is creating lot of issues, as all the indices of metrics-apm.app are rolled-over almost every other day

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [March 26, 2024, 5:20pm UTC](https://discuss.elastic.co/t/sudden-rollover-of-apm-indices/355473/3 "2024-03-26T17:20:41Z")

</div>

Hello,

You need to share your policies, both the original one and the one that you changed.

---

<div class="post-metadata">

**Author:** ![Harsh\_Jain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harsh_jain/32/132648_2.png) [@Harsh\_Jain](https://discuss.elastic.co/u/Harsh_Jain)\
**Post date:** [March 27, 2024, 9:05am UTC](https://discuss.elastic.co/t/sudden-rollover-of-apm-indices/355473/4 "2024-03-27T09:05:35Z")

</div>

Default policy

```auto
{
  "metrics-apm.app_metrics-default_policy": {
    "version": 1,
    "modified_date": "2024-03-26T10:16:30.889Z",
    "policy": {
      "phases": {
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_age": "30d",
              "max_size": "50gb"
            },
            "set_priority": {
              "priority": 100
            }
          }
        },
        "delete": {
          "min_age": "90d",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        }
      },
      "_meta": {
        "package": {
          "name": "apm"
        },
        "managed_by": "fleet",
        "managed": true
      }
    },
    "in_use_by": {
      "indices": [],
      "data_streams": [],
      "composable_templates": []
    }
  }
}

```

New Policy

```auto
{
  "new-metrics-apm.app_metrics-default_policy": {
    "version": 1,
    "modified_date": "2024-03-15T09:34:30.393Z",
    "policy": {
      "phases": {
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_age": "90d",
              "max_primary_shard_size": "50gb"
            },
            "set_priority": {
              "priority": 100
            }
          }
        },
        "delete": {
          "min_age": "180d",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        }
      }
    },
    "in_use_by": {
      "indices": [".ds-metrics-apm.app.test-default-2024.03.26-000015"],
      "data_streams": ["metrics-apm.app.test-default"],
      "composable_templates": [
        "metrics-apm.app"
      ]
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![Harsh\_Jain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harsh_jain/32/132648_2.png) [@Harsh\_Jain](https://discuss.elastic.co/u/Harsh_Jain)\
**Post date:** [April 1, 2024, 5:05am UTC](https://discuss.elastic.co/t/sudden-rollover-of-apm-indices/355473/5 "2024-04-01T05:05:16Z")

</div>

Please reply on this. It is causing lot of issue in my elasticsearch cluster.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [April 1, 2024, 12:26pm UTC](https://discuss.elastic.co/t/sudden-rollover-of-apm-indices/355473/6 "2024-04-01T12:26:39Z")

</div>

Hello,

I see no issues in the policy, the new policy should rollover after 90 days or if a primary shard reaches 50 gb, so not sure what is happening, you will need to provide more evidence.

What is the return of `GET _cat/shards`? This will show your shards to help understand how your indices looks like.

Also, check the elasticsearch logs and see if there is some logs about ILM.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 29, 2024, 12:27pm UTC](https://discuss.elastic.co/t/sudden-rollover-of-apm-indices/355473/7 "2024-04-29T12:27:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
