# \[Solved\] Indexes does not automatically created based on new date

**URL:** <https://discuss.elastic.co/t/solved-indexes-does-not-automatically-created-based-on-new-date/183640>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [May 31, 2019, 2:18am UTC](https://discuss.elastic.co/t/solved-indexes-does-not-automatically-created-based-on-new-date/183640 "2019-05-31T02:18:49Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![TsuWeiQuan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tsuweiquan/32/46252_2.png) [@TsuWeiQuan](https://discuss.elastic.co/u/TsuWeiQuan)\
**Post date:** [May 31, 2019, 2:18am UTC](https://discuss.elastic.co/t/solved-indexes-does-not-automatically-created-based-on-new-date/183640/1 "2019-05-31T02:18:49Z")

</div>

Hello,

I have this small problem where I realized my beats are not creating new indexes as the day goes by. All data are stored in a single index which doesn't look good and can be a problem it was deleted.

This problem arose when I changed my index name on the beats setting.  
As I prefer the naming format to be `<beat-type>-<hostname>-<date>` which is shown in the picture below.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/3/73599f092b24399b9d8136402b5753184f3f5a90.png)

All beats are sending logs directly into elasticsearch via modules and i have append these settings in each beat's yml file.

```
setup.ilm.enabled: auto
setup.ilm.rollover_alias: "filebeat-esnode1"
setup.ilm.pattern: "{now/d}-000001"
setup.template.overwrite: true

```

As you can see from the above picture, the metricbeat index contains 1.9Gb of data from 5 hosts with the date created on 23 May.

Is there a way to make these indexes create a new index after each day has passed Or this is actually the ideal way to be done?

---

<div class="post-metadata">

**Author:** ![faec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/faec/32/46988_2.png) [@faec](https://discuss.elastic.co/u/faec)\
**Post date:** [May 31, 2019, 7:48pm UTC](https://discuss.elastic.co/t/solved-indexes-does-not-automatically-created-based-on-new-date/183640/2 "2019-05-31T19:48:07Z")

</div>

Hi! Have you configured an index lifecycle policy, as described in [these docs](https://www.elastic.co/guide/en/elasticsearch/reference/7.1/getting-started-index-lifecycle-management.html)? If so, could you share your configuration?

---

<div class="post-metadata">

**Author:** ![TsuWeiQuan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tsuweiquan/32/46252_2.png) [@TsuWeiQuan](https://discuss.elastic.co/u/TsuWeiQuan)\
**Post date:** [June 3, 2019, 12:58am UTC](https://discuss.elastic.co/t/solved-indexes-does-not-automatically-created-based-on-new-date/183640/3 "2019-06-03T00:58:17Z")

</div>

I did not configure the index lifecycle policy, leaving them in the default state.  
Thank you for mentioning this, i did not know the purpose of ILM until now.  
PS i am still a newbie in elastic 🙂

After reading the Documentation, i understand that the indices will only be updated to the lastest after a phase change. However, the default state that was set was `max_size: 50gb` & `"max_age": "30d"` in the `hot` phase, `version 32`.

So I followed the [documentation](https://www.elastic.co/guide/en/elasticsearch/reference/master/_updates_to_executing_policies.html) to do an update the policies while indices are still writing.  
However, after countless of tries, running `GET filebeat-*/_ilm/explain` still shows the indices version at `32`.

```
{
  "indices" : {
    "filebeat-esnode2-2019.05.29-000001" : {
      "index" : "filebeat-esnode2-2019.05.29-000001",
      "managed" : true,
      "policy" : "filebeat-7.0.1",
      "lifecycle_date_millis" : 1559118013908,
      "phase" : "hot",
      "phase_time_millis" : 1559118014150,
      "action" : "rollover",
      "action_time_millis" : 1559118107765,
      "step" : "check-rollover-ready",
      "step_time_millis" : 1559118107765,
      "phase_execution" : {
        "policy" : "filebeat-7.0.1",
        "phase_definition" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_size" : "50gb",
              "max_age" : "30d"
            }
          }
        },
        "version" : 32,
        "modified_date_in_millis" : 1559118013357
      }
    },
    "filebeat-esnode1-2019.05.29-000001" : {
      "index" : "filebeat-esnode1-2019.05.29-000001",
      "managed" : true,
      "policy" : "filebeat-7.0.1",
      "lifecycle_date_millis" : 1559120066378,
      "phase" : "hot",
      "phase_time_millis" : 1559120066554,
      "action" : "rollover",
      "action_time_millis" : 1559120507394,
      "step" : "check-rollover-ready",
      "step_time_millis" : 1559120507394,
      "phase_execution" : {
        "policy" : "filebeat-7.0.1",
        "phase_definition" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_size" : "50gb",
              "max_age" : "30d"
            }
          }
        },
        "version" : 34,
        "modified_date_in_millis" : 1559120065826
      }
    }

```

Q1) Is it true that I have no way to change the settings until the criteria for rollover has been met?

Q2) Is there a recommended policy practice such as 2Gb/Index with a 90 days warm phase before deletion?

---

<div class="post-metadata">

**Author:** ![TsuWeiQuan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tsuweiquan/32/46252_2.png) [@TsuWeiQuan](https://discuss.elastic.co/u/TsuWeiQuan)\
**Post date:** [June 7, 2019, 6:44am UTC](https://discuss.elastic.co/t/solved-indexes-does-not-automatically-created-based-on-new-date/183640/4 "2019-06-07T06:44:17Z")

</div>

I have managed to solve this question via [another post](https://discuss.elastic.co/t/solved-how-can-i-do-a-manual-rollover-for-all-indices-to-the-newest-index-policy/184694) of mine.

Thank you!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2019, 6:44am UTC](https://discuss.elastic.co/t/solved-indexes-does-not-automatically-created-based-on-new-date/183640/5 "2019-07-05T06:44:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.