# Date in index name doesn't change on rollover on the next day

**URL:** https://discuss.elastic.co/t/date-in-index-name-doesnt-change-on-rollover-on-the-next-day/226701
**Category:** Elasticsearch
**Tags:** ilm-index-lifecycle-management
**Created:** [April 6, 2020, 11:53am UTC](https://discuss.elastic.co/t/date-in-index-name-doesnt-change-on-rollover-on-the-next-day/226701 "2020-04-06T11:53:11Z")
**Posts on this page:** 19
**Page:** 1

<div class="post-metadata">

### Author: ![EldrosKandar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eldroskandar/32/33188_2.png) [@EldrosKandar](https://discuss.elastic.co/u/EldrosKandar)
#### Post date: [April 6, 2020, 11:53am UTC](https://discuss.elastic.co/t/date-in-index-name-doesnt-change-on-rollover-on-the-next-day/226701/1 "2020-04-06T11:53:11Z")

</div>

For some reason, although the index naming convention is set in the Filebeat clients' configuration, the timestamp of the date isn't change when the index is rolled over on the next day.

For example let's say I had index "index-2020.04.02-000001" and "index-2020.04.02-000002" already created, the day following it created "index-2020.04.02-000003" instead of "index-2020.04.03-000001".

What do I miss?

---

<div class="post-metadata">

### Author: ![dakrone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dakrone/32/23351_2.png) [@dakrone](https://discuss.elastic.co/u/dakrone)
#### Post date: [April 13, 2020, 10:08pm UTC](https://discuss.elastic.co/t/date-in-index-name-doesnt-change-on-rollover-on-the-next-day/226701/2 "2020-04-13T22:08:24Z")

</div>

ES will automatically do the date math only if the initial index was created using date math, you can see what the index was created by with:

```auto
GET /<your-index>

```

And then look for the `provided_name` key, what does it say?

---

<div class="post-metadata">

### Author: ![EldrosKandar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eldroskandar/32/33188_2.png) [@EldrosKandar](https://discuss.elastic.co/u/EldrosKandar)
#### Post date: [April 14, 2020, 6:54am UTC](https://discuss.elastic.co/t/date-in-index-name-doesnt-change-on-rollover-on-the-next-day/226701/3 "2020-04-14T06:54:56Z")

</div>

The provided name for `provided_name` give the same as the index name, i.e for the index "index-2020.04.02-000001" it is `index-2020.04.02-000001`.

---

<div class="post-metadata">

### Author: ![dakrone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dakrone/32/23351_2.png) [@dakrone](https://discuss.elastic.co/u/dakrone)
#### Post date: [April 14, 2020, 3:05pm UTC](https://discuss.elastic.co/t/date-in-index-name-doesnt-change-on-rollover-on-the-next-day/226701/4 "2020-04-14T15:05:14Z")

</div>

Okay, in that case it looks like the first index was not created with date math, rollover will only perpetuate the date math in the name if the initial index was created with date math (it would be named something like `<index-{now/d}-000001>` if it were created with date math.

In order to migrate it, it depends on what version of ES you're running, what version are you using? (Alternatively, if you're just testing, you can start over, but I understand that's not feasible with live data)

---

<div class="post-metadata">

### Author: ![EldrosKandar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eldroskandar/32/33188_2.png) [@EldrosKandar](https://discuss.elastic.co/u/EldrosKandar)
#### Post date: [April 15, 2020, 6:27am UTC](https://discuss.elastic.co/t/date-in-index-name-doesnt-change-on-rollover-on-the-next-day/226701/5 "2020-04-15T06:27:39Z")

</div>

Well, this has probably happened, after some re-indexing. Is it possible to create an index with a REST call using date math?

I'm currently on 7.6

---

<div class="post-metadata">

### Author: ![EldrosKandar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eldroskandar/32/33188_2.png) [@EldrosKandar](https://discuss.elastic.co/u/EldrosKandar)
#### Post date: [April 21, 2020, 7:41am UTC](https://discuss.elastic.co/t/date-in-index-name-doesnt-change-on-rollover-on-the-next-day/226701/6 "2020-04-21T07:41:06Z")

</div>

@dakrone How do I migrate to an index with date math in the name on 7.6?

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [May 5, 2020, 5:49am UTC](https://discuss.elastic.co/t/date-in-index-name-doesnt-change-on-rollover-on-the-next-day/226701/7 "2020-05-05T05:49:26Z")

</div>

ILM is built around the date in the index name being the date the first index in the policy was created. There's currently no way to change that to have subsequent index names created with the date it was then created on.  
If it's something you would like, it'd be worth creating a feature request so we can track it 🙂

> [@EldrosKandar](#):
>
> Is it possible to create an index with a REST call using date math?

If the client you are using can translate that into an actual index name, yes. Elasticsearch cannot calculate that for you via the REST API.

---

<div class="post-metadata">

### Author: ![EldrosKandar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eldroskandar/32/33188_2.png) [@EldrosKandar](https://discuss.elastic.co/u/EldrosKandar)
#### Post date: [May 5, 2020, 7:29am UTC](https://discuss.elastic.co/t/date-in-index-name-doesnt-change-on-rollover-on-the-next-day/226701/8 "2020-05-05T07:29:44Z")

</div>

So you are basically saying that the only way would be delete all the productive indices and let the Filebeat client generate the next index with the date math, correct?

That's not very realistic for me, as I don't want to delete all the data, considering those are productive data.

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [May 5, 2020, 8:33am UTC](https://discuss.elastic.co/t/date-in-index-name-doesnt-change-on-rollover-on-the-next-day/226701/9 "2020-05-05T08:33:49Z")

</div>

What's the driver behind wanting date based indices?

---

<div class="post-metadata">

### Author: ![EldrosKandar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eldroskandar/32/33188_2.png) [@EldrosKandar](https://discuss.elastic.co/u/EldrosKandar)
#### Post date: [May 5, 2020, 10:09am UTC](https://discuss.elastic.co/t/date-in-index-name-doesnt-change-on-rollover-on-the-next-day/226701/10 "2020-05-05T10:09:50Z")

</div>

The ILM is configured so, it rollover each day or when it reaches 50GB.

We use the stack to be able to search our logs. The driver would be to see in one go which indices contains the logging data of specific day.

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [May 5, 2020, 10:19am UTC](https://discuss.elastic.co/t/date-in-index-name-doesnt-change-on-rollover-on-the-next-day/226701/11 "2020-05-05T10:19:04Z")

</div>

It has been a pattern we have used for such a long time now, so it makes sense. But ILM removes that requirement, and Kibana now uses the [`field_caps`](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/search-field-caps.html) API to get that info from an index when it needs to.

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [May 5, 2020, 12:36pm UTC](https://discuss.elastic.co/t/date-in-index-name-doesnt-change-on-rollover-on-the-next-day/226701/12 "2020-05-05T12:36:38Z")

</div>

I do not think Kibana has used field caps in quite some time. A while back querying indices with no matches was made much more efficient so I think it now just queries all matching indices.

---

<div class="post-metadata">

### Author: ![dakrone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dakrone/32/23351_2.png) [@dakrone](https://discuss.elastic.co/u/dakrone)
#### Post date: [May 5, 2020, 3:29pm UTC](https://discuss.elastic.co/t/date-in-index-name-doesnt-change-on-rollover-on-the-next-day/226701/13 "2020-05-05T15:29:53Z")

</div>

> [@warkolm](#):
>
> What's the driver behind wanting date based indices?

Date-based indices are much easier to deal with as an administrator when doing things like trying to figure out what index to restore from a snapshot, having `index-2018-05-21-000234` gives a better idea of when the index was created and what data it has than `index-000234`.

---

<div class="post-metadata">

### Author: ![EldrosKandar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eldroskandar/32/33188_2.png) [@EldrosKandar](https://discuss.elastic.co/u/EldrosKandar)
#### Post date: [May 6, 2020, 4:42am UTC](https://discuss.elastic.co/t/date-in-index-name-doesnt-change-on-rollover-on-the-next-day/226701/14 "2020-05-06T04:42:58Z")

</div>

So where does that leave me? As I said dakrone hinted that there is way to migrate my index with the desired behaviour, depending on the version of the ES stack I'm using. (btw, we recently updated to 7.6.2). Refer to the quote below to see what I mean. And as I am productive, I'm loathe to throw simply all my data away.

> [@dakrone](#):
>
> In order to migrate it, it depends on what version of ES you're running, what version are you using? (Alternatively, if you're just testing, you can start over, but I understand that's not feasible with live data)

---

<div class="post-metadata">

### Author: ![dakrone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dakrone/32/23351_2.png) [@dakrone](https://discuss.elastic.co/u/dakrone)
#### Post date: [May 8, 2020, 9:21pm UTC](https://discuss.elastic.co/t/date-in-index-name-doesnt-change-on-rollover-on-the-next-day/226701/15 "2020-05-08T21:21:40Z")

</div>

@EldrosKandar here are some steps for it:

1. Stop ILM

```auto
POST /_ilm/stop

```

1. Create the new index with date math (don't change the alias yet)

(replace 999 with the number needed, whichever is the next number in your sequence):

```auto
PUT %3Cindex-%7Bnow%2Fd%7D-000999%3E

```

1. Update the aliases to correct the write\_alias pointer

(replace the `index-2020.07.30-000999` index name with the index created in step 2 and replace `index-2020.06.19-000NNN` with the latest index that exists, replace `youralias` with your alias name):

```auto
POST /_aliases
{
  "actions" : [
    {
      "add" : {
        "index" : "index-2020.07.30-000999",
        "alias" : "youralias",
        "is_write_index" : true
      }
    },
    {
      "add" : {
        "index" : "index-2020.06.19-000NNN",
        "alias" : "youralias",
        "is_write_index" : false
      }
    }
  ]
}

```

1. Update the first index to say that indexing is complete

(skip the rollover since you already created the next index), make sure to replace the NNN with the correct number!

```auto
PUT /logstash-atl_fdmvcc-2019.06.19-000NNN/_settings
{
  "index.lifecycle.indexing_complete": true
}

```

1. Start ILM

```auto
POST /_ilm/start

```

1. Verify with ILM explain that everything looks copacetic

---

<div class="post-metadata">

### Author: ![EldrosKandar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eldroskandar/32/33188_2.png) [@EldrosKandar](https://discuss.elastic.co/u/EldrosKandar)
#### Post date: [May 12, 2020, 9:51am UTC](https://discuss.elastic.co/t/date-in-index-name-doesnt-change-on-rollover-on-the-next-day/226701/16 "2020-05-12T09:51:26Z")

</div>

Thanks a lot! I've applied it today, now let's see if it works when it creates the next index.

This ought to be better documented, though. Would it be a case where I should open an issue?

---

<div class="post-metadata">

### Author: ![EldrosKandar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eldroskandar/32/33188_2.png) [@EldrosKandar](https://discuss.elastic.co/u/EldrosKandar)
#### Post date: [May 15, 2020, 4:22am UTC](https://discuss.elastic.co/t/date-in-index-name-doesnt-change-on-rollover-on-the-next-day/226701/17 "2020-05-15T04:22:43Z")

</div>

I had to do a re-index yesterday, due to another issue I had, and I adapter your index creation call formula in the re-index call. Last night a new index was created and it took the newest date.

Thanks @dakrone for your support and your patience with me.

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [May 15, 2020, 4:24am UTC](https://discuss.elastic.co/t/date-in-index-name-doesnt-change-on-rollover-on-the-next-day/226701/18 "2020-05-15T04:24:09Z")

</div>

Great to hear!  
If you would like to open an issue here - [https://github.com/elastic/elasticsearch/issues/new/choose](https://github.com/elastic/elasticsearch/issues/new/choose)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 12, 2020, 4:24am UTC](https://discuss.elastic.co/t/date-in-index-name-doesnt-change-on-rollover-on-the-next-day/226701/19 "2020-06-12T04:24:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
