# APM index failing ILM rollover

**URL:** <https://discuss.elastic.co/t/apm-index-failing-ilm-rollover/197823>\
**Category:** APM\
**Created:** [September 3, 2019, 10:40am UTC](https://discuss.elastic.co/t/apm-index-failing-ilm-rollover/197823 "2019-09-03T10:40:29Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![ethranes](https://avatars.discourse-cdn.com/v4/letter/e/e56c9b/32.png) [@ethranes](https://discuss.elastic.co/u/ethranes)\
**Post date:** [September 3, 2019, 10:40am UTC](https://discuss.elastic.co/t/apm-index-failing-ilm-rollover/197823/1 "2019-09-03T10:40:29Z")

</div>

Hi, I have my APM setup, and I thought I setit up correctly for ILM, it rolled over once from 0000001 to 0000002 but now the index won't rollover, I get the following error

illegal\_argument\_exception: setting [index.lifecycle.rollover\_alias] for index [apm-6.7.1-span-000002] is empty or not defined

Could anyone offer any help or advice on debugging and getting my rollovers working?

---

<div class="post-metadata">

**Author:** ![jalvz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jalvz/32/45613_2.png) [@jalvz](https://discuss.elastic.co/u/jalvz)\
**Post date:** [September 3, 2019, 11:38am UTC](https://discuss.elastic.co/t/apm-index-failing-ilm-rollover/197823/2 "2019-09-03T11:38:27Z")

</div>

Hello!

Sorry to hear. I am assuming you are using apm-server and elasticsearch 7.3, is that correct?  
Can we see your apm-server.yml, index templates, and ilm policies?

---

<div class="post-metadata">

**Author:** ![ethranes](https://avatars.discourse-cdn.com/v4/letter/e/e56c9b/32.png) [@ethranes](https://discuss.elastic.co/u/ethranes)\
**Post date:** [September 4, 2019, 10:29am UTC](https://discuss.elastic.co/t/apm-index-failing-ilm-rollover/197823/3 "2019-09-04T10:29:47Z")

</div>

Hi, yeah we're using an apm server, our elastic search version is 6.7.1. Our index template looks like this

```
{
  "apm-6.7.1-span" : {
    "order" : 2,
    "index_patterns" : [
      "apm-6.7.1-span-*"
    ],
    "settings" : {
      "index" : {
        "lifecycle" : {
          "name" : "apm-6.7.1-span",
          "rollover_alias" : "apm-6.7.1-span"
        }
      }
    },
    "mappings" : { },
    "aliases" : { }
  }
}

```

And this is the ilm policy

```
{
  "policy": "apm-6.7.1-span",
  "phase_definition": {
    "min_age": "0ms",
    "actions": {
      "rollover": {
        "max_size": "100mb",
        "max_age": "1d"
      },
      "set_priority": {
        "priority": 100
      }
    }
  },
  "version": 3,
  "modified_date_in_millis": 1567589561158
}

```

I've been able to clear the error that I was getting yesterday by creating a new alias;

```
PUT apm-6.7.1-span-000001 
{
  "aliases": {
    "apm-6.7.1-span":{
      "is_write_index": true 
    }
  }
}

```

So now the error message is gone. The issue that I have now is that you can see I've set the index to rollover after 100mb. It's past that threshold, and it's current action shows rollover, but it doesn't look like it's doing it, it's now up to 130mb and is still trying to do the rollover

---

<div class="post-metadata">

**Author:** ![ethranes](https://avatars.discourse-cdn.com/v4/letter/e/e56c9b/32.png) [@ethranes](https://discuss.elastic.co/u/ethranes)\
**Post date:** [September 4, 2019, 10:46am UTC](https://discuss.elastic.co/t/apm-index-failing-ilm-rollover/197823/4 "2019-09-04T10:46:50Z")

</div>

The index has rolled over now, is there any indication of how long a rollover takes? it doesn't seem right that an index will get to double (200mb in this case) it's policy limit before the rollover completes and opens a new index

---

<div class="post-metadata">

**Author:** ![jalvz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jalvz/32/45613_2.png) [@jalvz](https://discuss.elastic.co/u/jalvz)\
**Post date:** [September 4, 2019, 11:44am UTC](https://discuss.elastic.co/t/apm-index-failing-ilm-rollover/197823/5 "2019-09-04T11:44:18Z")

</div>

Glad that you found the issue, you needed that alias indeed!

`max_size` should be taken as an estimate, and not as an exact value. People often use much larger values for that setting, and in the context of many gigabytes 100 megs deviation doesn't make much of a difference.

As a matter of fact (unless you were only testing), If you want to avoid too much fragmentation you might consider bumping up that quite a bit: somewhere between 10 and 50 Gb is very reasonable.

A rollover doesn't take much time, but it kicks in every 10 minutes by default. You can change it like this if you wish:

```auto
{
  "transient": {
    "indices.lifecycle.poll_interval": "1m" 
  }
}

```

Hope this helps!

---

<div class="post-metadata">

**Author:** ![ethranes](https://avatars.discourse-cdn.com/v4/letter/e/e56c9b/32.png) [@ethranes](https://discuss.elastic.co/u/ethranes)\
**Post date:** [September 4, 2019, 11:51am UTC](https://discuss.elastic.co/t/apm-index-failing-ilm-rollover/197823/6 "2019-09-04T11:51:05Z")

</div>

Thanks for your response, yeah the 100mb was just for testing the rollover, we wouldn't have such a strict size policy as that, seeing as we'll be doing 50gb rollovers in a cluster that has about 8tb of space, so that's good news. Thanks for the help

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 25, 2019, 7:51am UTC](https://discuss.elastic.co/t/apm-index-failing-ilm-rollover/197823/7 "2019-09-25T07:51:13Z")

</div>

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.