# Index rollover issues

**URL:** <https://discuss.elastic.co/t/index-rollover-issues/290669>\
**Category:** Elasticsearch\
**Tags:** ilm-index-lifecycle-management\
**Created:** [December 1, 2021, 1:22pm UTC](https://discuss.elastic.co/t/index-rollover-issues/290669 "2021-12-01T13:22:20Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![sandeepmuthathi](https://avatars.discourse-cdn.com/v4/letter/s/48db29/32.png) [@sandeepmuthathi](https://discuss.elastic.co/u/sandeepmuthathi)\
**Post date:** [December 1, 2021, 1:22pm UTC](https://discuss.elastic.co/t/index-rollover-issues/290669/1 "2021-12-01T13:22:20Z")

</div>

Suppose I'm creating the ES index using beat system or logstash with date time interpolation.

1. how do I configure rolling over of index if size reaches at certain GBs in a single day? I did the following but I'm doing something wrong.  
I created an ILM and then an index template and linked the ILM and let logstash create the new index. But index gives the following error. Any idea?  
illegal\_argument\_exception: index.lifecycle.rollover\_alias [test-index] does not point to index [test-index-2021.12.01]

2. What is the best practice? Create the index from beat/logstash with date or configuring entire rollover stuff from ES itself?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [December 1, 2021, 11:41pm UTC](https://discuss.elastic.co/t/index-rollover-issues/290669/2 "2021-12-01T23:41:16Z")

</div>

Welcome to our community! 😃

Please share your policy and index template.

---

<div class="post-metadata">

**Author:** ![sandeepmuthathi](https://avatars.discourse-cdn.com/v4/letter/s/48db29/32.png) [@sandeepmuthathi](https://discuss.elastic.co/u/sandeepmuthathi)\
**Post date:** [December 2, 2021, 6:27am UTC](https://discuss.elastic.co/t/index-rollover-issues/290669/3 "2021-12-02T06:27:22Z")

</div>

Thank you.

Here you go.

Index settings:

```auto
{
  "test-index" : {
    "settings" : {
      "index" : {
        "lifecycle" : {
          "name" : "test-ilm-policy"
        },
        "routing" : {
          "allocation" : {
            "include" : {
              "_tier_preference" : "data_content"
            }
          }
        },
        "number_of_shards" : "1",
        "provided_name" : "test-index",
        "creation_date" : "1638365760565",
        "priority" : "100",
        "number_of_replicas" : "1",
        "uuid" : "4LdlNUg-Q9WB81heA4b94Q",
        "version" : {
          "created" : "7150299"
        }
      }
    }
  }

ILM settings : 

{
  "test-ilm-policy" : {
    "version" : 3,
    "modified_date" : "2021-12-01T13:32:49.370Z",
    "policy" : {
      "phases" : {
        "hot" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_size" : "300kb",
              "max_primary_shard_size" : "10gb",
              "max_age" : "1d"
            },
            "set_priority" : {
              "priority" : 100
            }
          }
        },
        "delete" : {
          "min_age" : "2d",
          "actions" : {
            "delete" : {
              "delete_searchable_snapshot" : true
            }
          }
        }
      }
    },
    "in_use_by" : {
      "indices" : [
        "test-index",
      ],
      "data_streams" : [],
      "composable_templates" : []
    }
  }
}

Template: 

{
  "my_template" : {
    "order" : 0,
    "index_patterns" : [
      “test-index-*”
    ],
    "settings" : {
      "index" : {
        "lifecycle" : {
          "name" : “test-ilm-policy",
          "rollover_alias" : "test-index”
        },
        "number_of_shards" : "1",
        "number_of_replicas" : "1"
      }
    },
    "mappings" : { },
    "aliases" : { }
  }
}

```

---

<div class="post-metadata">

**Author:** ![sandeepmuthathi](https://avatars.discourse-cdn.com/v4/letter/s/48db29/32.png) [@sandeepmuthathi](https://discuss.elastic.co/u/sandeepmuthathi)\
**Post date:** [December 3, 2021, 7:40am UTC](https://discuss.elastic.co/t/index-rollover-issues/290669/4 "2021-12-03T07:40:30Z")

</div>

@warkolm I have shared the details that you asked.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 3, 2021, 3:38pm UTC](https://discuss.elastic.co/t/index-rollover-issues/290669/5 "2021-12-03T15:38:00Z")

</div>

> [@sandeepmuthathi](#):
>
> . But index gives the following error. Any idea?  
> illegal\_argument\_exception: index.lifecycle.rollover\_alias [test-index] does not point to index [test-index-2021.12.01]

I suspect perhaps You need to create the initial managed index that maps the writer alias to a concrete index.

See [Here](https://www.elastic.co/guide/en/elasticsearch/reference/current/set-up-lifecycle-policy.html#create-initial-index)

Yours would look something like

```auto
PUT test-index-2021.12.01
{
  "aliases": {
    "test-index":{
      "is_write_index": true 
    }
  }
}

```

Also

```auto
            "rollover" : {
              "max_size" : "300kb", <!----- This is not going to work well
              "max_primary_shard_size" : "10gb",
              "max_age" : "1d"

```

ILM is mean to work on the scale of GBs etc. so it will not rollover exactly on 300KB etc. I have written a bit about that [here](https://discuss.elastic.co/t/ilm-doesnt-work/284285/3)

Also you will never want 300KB indices that is very small and inefficient.

> [@sandeepmuthathi](#):
>
> What is the best practice? Create the index from beat/logstash with date or configuring entire rollover stuff from ES itself?

Well typically I would suggest starting with the defaults from say filebeat where all this is already configured... and works out of the box... get used to how elastic works.

Today we suggest Shard Sized based Rollover ... Time Based / Daily has some usefulness but can end up with many small indices and shards which can be wasteful.

And of course there are some nice docs [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html) and [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/size-your-shards.html)

---

<div class="post-metadata">

**Author:** ![sandeepmuthathi](https://avatars.discourse-cdn.com/v4/letter/s/48db29/32.png) [@sandeepmuthathi](https://discuss.elastic.co/u/sandeepmuthathi)\
**Post date:** [December 13, 2021, 8:35am UTC](https://discuss.elastic.co/t/index-rollover-issues/290669/6 "2021-12-13T08:35:56Z")

</div>

@stephenb Thank you for your answer. 300kb, I set for testing rollover, for faster rollover. please ignore that.

I got your point, what I'm trying to do is set roll over via filebeat(using date) and also set additional roll over via shrads or size. How can I achieve that?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 13, 2021, 3:50pm UTC](https://discuss.elastic.co/t/index-rollover-issues/290669/7 "2021-12-13T15:50:26Z")

</div>

It does not really work that way... either you are using ILM or date based rollover in filebeat with index names not both.

The closest is to use ILM set Max Age 1d and a Max Shard Size.. that way it will either roll over 1 a day or or busier indices rollover at the shard size.

---

<div class="post-metadata">

**Author:** ![sandeepmuthathi](https://avatars.discourse-cdn.com/v4/letter/s/48db29/32.png) [@sandeepmuthathi](https://discuss.elastic.co/u/sandeepmuthathi)\
**Post date:** [December 14, 2021, 2:31pm UTC](https://discuss.elastic.co/t/index-rollover-issues/290669/8 "2021-12-14T14:31:19Z")

</div>

Thank you @stephenb

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 11, 2022, 2:32pm UTC](https://discuss.elastic.co/t/index-rollover-issues/290669/9 "2022-01-11T14:32:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.