# Elasticsearch Create Wrong time based Index

**URL:** <https://discuss.elastic.co/t/elasticsearch-create-wrong-time-based-index/272785>\
**Category:** Elasticsearch\
**Tags:** ilm-index-lifecycle-management\
**Created:** [May 12, 2021, 8:42am UTC](https://discuss.elastic.co/t/elasticsearch-create-wrong-time-based-index/272785 "2021-05-12T08:42:59Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Aniket\_Pant](https://avatars.discourse-cdn.com/v4/letter/a/77aa72/32.png) [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\
**Post date:** [May 12, 2021, 8:42am UTC](https://discuss.elastic.co/t/elasticsearch-create-wrong-time-based-index/272785/1 "2021-05-12T08:42:59Z")

</div>

Using ELK 7.11  
we have 11 index out of these we have one index called log-wlb-sysmon.It created its first index on 2021.04.06(log-wlb-sysmon-2021.04.06-000001) , we are using ILM policy for all the indexes  
It created the another index on 19th April (log-wlb-sysmon-2021.04.19-000002) because we have set `"max_age": "30d", "max_size": "50gb"` , after some period of time we faced some mapping problem in this index _so we decided to delete our old index(log-wlb-sysmon-2021.04.19-000002) and recreate this index so we did this_

```auto
PUT log-wlb-sysmon-2021.04.26-000002
{
 "aliases": {
   "log-wlb-sysmon": {
      "is_write_index": true 
   }
 }
}

```

from then it was working fine both the indices(log-wlb-sysmon-2021.04.06-000001 and (log-wlb-sysmon-2021.04.26-000002)

Now we've notice that it created index log-wlb-sysmon-2021.04.26-000003 on 7th april , it should create index like log-wlb-sysmon-2021.05.07-000003

```auto
[2021-05-07T16:30:36,725][INFO][o.e.c.m.MetadataCreateIndexService] [em2] [log-wlb-sysmon-2021.04.26-000003] creating index, cause [rollover_index], templates [winlogbeat_sysmon], shards [1]/[1]
[2021-05-07T16:30:36,767][INFO][o.e.x.i.IndexLifecycleTransition] [em2] moving index [log-wlb-sysmon-2021.04.26-000003] from [null] to [{"phase":"new","action":"complete","name":"complete"}] in policy [winlogbeat_sysmon_policy]
[2021-05-07T16:30:36,845][INFO][o.e.x.i.IndexLifecycleTransition] [em2] moving index [log-wlb-sysmon-2021.04.26-000002] from [{"phase":"hot","action":"rollover","name":"attempt-rollover"}] to [{"phase":"hot","action":"rollover","name":"wait-for-active-shards"}] in policy [winlogbeat_sysmon_policy]
[2021-05-07T16:30:36,880][INFO][o.e.c.m.MetadataMappingService] [em2] [log-wlb-sysmon-2021.04.26-000003/QSzhCbMPRQ2Yynils14EFA] update_mapping [_doc]
[2021-05-07T16:30:36,920][INFO][o.e.x.i.IndexLifecycleTransition] [em2] moving index [log-wlb-sysmon-2021.04.26-000003] from [{"phase":"new","action":"complete","name":"complete"}] to [{"phase":"hot","action":"set_priority","name":"set_priority"}] in policy [winlogbeat_sysmon_policy]
[2021-05-07T16:30:36,989][INFO][o.e.c.m.MetadataMappingService] [em2] [log-wlb-sysmon-2021.04.26-000003/QSzhCbMPRQ2Yynils14EFA] update_mapping [_doc]
[2021-05-07T16:30:36,991][INFO][o.e.c.m.MetadataMappingService] [em2] [log-wlb-sysmon-2021.04.26-000003/QSzhCbMPRQ2Yynils14EFA] update_mapping [_doc]
[2021-05-07T16:30:37,032][INFO][o.e.x.i.IndexLifecycleTransition] [em2] moving index [log-wlb-sysmon-2021.04.26-000002] from [{"phase":"hot","action":"rollover","name":"wait-for-active-shards"}] to [{"phase":"hot","action":"rollover","name":"update-rollover-lifecycle-date"}] in policy [winlogbeat_sysmon_policy]
[2021-05-07T16:30:37,033][INFO][o.e.x.i.IndexLifecycleTransition] [em2] moving index [log-wlb-sysmon-2021.04.26-000002] from [{"phase":"hot","action":"rollover","name":"update-rollover-lifecycle-date"}] to [{"phase":"hot","action":"rollover","name":"set-indexing-complete"}] in policy [winlogbeat_sysmon_policy]
[2021-05-07T16:30:37,100][INFO][o.e.c.m.MetadataMappingService] [em2] [log-wlb-sysmon-2021.04.26-000003/QSzhCbMPRQ2Yynils14EFA] update_mapping [_doc]

```

**log-wlb-sysmon index detail**

```auto
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open log-wlb-sysmon-2021.04.26-000002 p5ROhVjwT0Gpl3pgwu2bRA 1 1 84829663 79510 100gb 50gb
green open log-wlb-sysmon-2021.04.06-000001 IAHprh30T6-rh2hxB67TLA 1 1 84049138 52926 100gb 50gb
green open log-wlb-sysmon-2021.04.26-000003 QSzhCbMPRQ2Yynils14EFA 1 1 39651126 67857 46.6gb 23.3gb

```

**logstash pipeline file for this index**

```auto
 elasticsearch {
      hosts => ["https://xx.xx.xx.xx:9200","https://xx.xx.xx.xx:9200"]
      cacert => '/etc/logstash/certs/ca/ca.crt'
      user => "logstash_user"
      password => "password@xxx"
      document_id => "%{[@metadata][log_hash]}"
      manage_template => false
      ilm_rollover_alias => "log-wlb-sysmon"
      ilm_pattern => "{now/d}-000001"
      ilm_enabled => "true"
      ilm_policy => "winlogbeat_sysmon_policy"

```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 12, 2021, 11:20pm UTC](https://discuss.elastic.co/t/elasticsearch-create-wrong-time-based-index/272785/2 "2021-05-12T23:20:25Z")

</div>

> [@Aniket\_Pant](#):
>
> Now we've notice that it created index log-wlb-sysmon-2021.04.26-000003 on 7th april , it should create index like log-wlb-sysmon-2021.05.07-000003

That's not how ILM works, the timestamp in the index name is the date that the ILM policy created its first index.

---

<div class="post-metadata">

**Author:** ![Aniket\_Pant](https://avatars.discourse-cdn.com/v4/letter/a/77aa72/32.png) [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\
**Post date:** [May 13, 2021, 10:44am UTC](https://discuss.elastic.co/t/elasticsearch-create-wrong-time-based-index/272785/3 "2021-05-13T10:44:42Z")

</div>

I don't know i am right or wrong because i am using the ilm concept from past 1 month,i have other indices also apart from log-wlb-sysmon

```auto
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open log-pb-flow-2021.04.05-000001 _HTx3o3iTEW25cP-anki-A 1 1 37767666 0 39.2gb 19.6gb
green open log-pb-tls-2021.05.05-000002 wiWn9yhrQ8yHwbV_zSLmsw 1 1 86550 0 361.2mb 179.5mb
green open log-pb-dns-2021.04.05-000001 y51OL9YBTBiZ4bFY-Kdupg 1 1 236493 0 455mb 226.7mb
green open log-pb-http-2021.05.05-000002 6PcQu2Q6Q0q5-yqotcGD6Q 1 1 18021 0 42.1mb 21.3mb
green open log-pb-tls-2021.04.05-000001 ZGu9rPQiT765hT_L0wun5g 1 1 254845 0 969.4mb 489.4mb
green open log-pb-icmp-2021.04.05-000001 NwzA1kw-TFm37IRcKknAGQ 1 1 437700 0 337.2mb 168.5mb
green open log-pb-icmp-2021.05.05-000002 liHOxFn6T5GMojxEunTNRA 1 1 84208 0 94.5mb 59.1mb
green open log-pb-http-2021.04.05-000001 OjapOmTCQa2k4xxgzA3-4w 1 1 159927 0 192.3mb 96.2mb
green open log-pb-flow-2021.05.05-000002 CkJzzRBxQ1m1BDD3Rkqcjg 1 1 9342075 0 9.7gb 4.8gb
green open log-pb-dns-2021.05.05-000002 AA7i7xrdSsCfA73gBJncSA 1 1 64963 0 144.6mb 72mb

```

You can see that month is changing but in log-wlb-sysmon index case it is was not changing, it created the new index but with same date .

---

<div class="post-metadata">

**Author:** ![andreidan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andreidan/32/78167_2.png) [@andreidan](https://discuss.elastic.co/u/andreidan)\
**Post date:** [May 13, 2021, 2:12pm UTC](https://discuss.elastic.co/t/elasticsearch-create-wrong-time-based-index/272785/4 "2021-05-13T14:12:13Z")

</div>

Thanks for using Elasticsearch and ILM.

[Date math](https://www.elastic.co/guide/en/elasticsearch/reference/current/date-math-index-names.html) on rollover is based on a hidden index setting called `index.provided_name`.

When you executed

```auto
PUT log-wlb-sysmon-2021.04.26-000002
{
 "aliases": {
   "log-wlb-sysmon": {
      "is_write_index": true 
   }
 }
}

```

the index was created as a regular index that has a date in its name, but date math was not configured.

Unfortunately there is no way to enable date math for an index after it's been created (`index.provided_name` is a private, internal setting). However you can create a new index with date math configured and then switch the `is_write_index` alias config from `log-wlb-sysmon-2021.04.26-000002` to the new index you created.

eg (please adapt the indices names to match your current state)

```auto
// this will create log-wlb-sysmon-2021.04.26-000001
PUT %3Clog-wlb-sysmon-%7Bnow%2Fd%7D-000001%3E 
{
"aliases": {
   "log-wlb-sysmon": {
      "is_write_index": false // keep it false for now as log-wlb-sysmon-2021.04.26-000002 is still the write_index for this alias
   }
 }
}

// swap the write alias from the previous index to the newly created one
POST _aliases
{
  "actions": [
    {
      "add": {
        "index": "log-wlb-sysmon-2021.04.26-000002",
        "alias": "log-wlb-sysmon",
        "is_write_index": false
      }
    }, {
      "add": {
        "index": "log-wlb-sysmon-2021.05.13-000001",
        "alias": "log-wlb-sysmon",
        "is_write_index": true
      }
    }
  ]
}

```

You can verify the new index has date math configured by issuing a  
`GET log-wlb-sysmon-2021.04.26-000001/_settings` and looking for the setting

```auto
"index": {
   ... 
        "provided_name": "<log-wlb-sysmon-{now/d}-000001>",
   ...
}

```

---

<div class="post-metadata">

**Author:** ![Aniket\_Pant](https://avatars.discourse-cdn.com/v4/letter/a/77aa72/32.png) [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\
**Post date:** [May 13, 2021, 3:47pm UTC](https://discuss.elastic.co/t/elasticsearch-create-wrong-time-based-index/272785/5 "2021-05-13T15:47:21Z")

</div>

Hey @andreidan i understand your point but i want to know that in ilm policy we have `"max_age": "30d", "max_size": "50gb"` for log-wlb-sysmon index , it will create new index after 20 days because in 20 days it has 100gb of data(replica including) my question is will it create index on time based or this log-wlb-sysmon-2021.04.26-000003 if it create index like this  
Then what we have to do to change this permanently so that in future we don't have to deal with this

---

<div class="post-metadata">

**Author:** ![andreidan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andreidan/32/78167_2.png) [@andreidan](https://discuss.elastic.co/u/andreidan)\
**Post date:** [May 13, 2021, 4:32pm UTC](https://discuss.elastic.co/t/elasticsearch-create-wrong-time-based-index/272785/6 "2021-05-13T16:32:57Z")

</div>

`log-wlb-sysmon-2021.04.26-000002` doesn't have date math configured at the moment so on the next rollover operation index `log-wlb-sysmon-2021.04.26-000003` will be created.

In order for date math to be enabled (and for rollover to update the date in the index name) you'll have to create a new index with date math enabled (with ILM configured for it, presumbaly by an index template? so no extra action should be required for this) and shift the alias with `is_write_index` to the newly created index as described in the [previous comment](https://discuss.elastic.co/t/elasticsearch-create-wrong-time-based-index/272785/4)

---

<div class="post-metadata">

**Author:** ![Aniket\_Pant](https://avatars.discourse-cdn.com/v4/letter/a/77aa72/32.png) [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\
**Post date:** [May 13, 2021, 4:51pm UTC](https://discuss.elastic.co/t/elasticsearch-create-wrong-time-based-index/272785/7 "2021-05-13T16:51:59Z")

</div>

I will test this

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 15, 2021, 5:29am UTC](https://discuss.elastic.co/t/elasticsearch-create-wrong-time-based-index/272785/9 "2021-06-15T05:29:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.