# Data stream indices not moving from frozen to delete state

**URL:** <https://discuss.elastic.co/t/data-stream-indices-not-moving-from-frozen-to-delete-state/304622>\
**Category:** Elasticsearch\
**Tags:** ilm-index-lifecycle-management\
**Created:** [May 12, 2022, 8:57pm UTC](https://discuss.elastic.co/t/data-stream-indices-not-moving-from-frozen-to-delete-state/304622 "2022-05-12T20:57:37Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![elastictor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/elastictor/32/105581_2.png) [@elastictor](https://discuss.elastic.co/u/elastictor)\
**Post date:** [May 12, 2022, 8:57pm UTC](https://discuss.elastic.co/t/data-stream-indices-not-moving-from-frozen-to-delete-state/304622/1 "2022-05-12T20:57:37Z")

</div>

I have created a data stream with ILM with phases ranging from Hot to Deleted.

The backing indices are successfully moved into Frozen state but never continue into the Deleted phase.

Why are the indices not moving to deleted state?

Here is the lifecycle policy:

```auto
{
  "filebeat" : {
    "version" : 11,
    "modified_date" : "2022-05-11T10:05:49.653Z",
    "policy" : {
      "phases" : {
        "frozen" : {
          "min_age" : "70m",
          "actions" : {
            "searchable_snapshot" : {
              "snapshot_repository" : "filebeat",
              "force_merge_index" : true
            }
          }
        },
        "warm" : {
          "min_age" : "30m",
          "actions" : {
            "set_priority" : {
              "priority" : 50
            }
          }
        },
        "cold" : {
          "min_age" : "50m",
          "actions" : {
            "set_priority" : {
              "priority" : 0
            }
          }
        },
        "hot" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_size" : "50gb",
              "max_age" : "20m"
            }
          }
        },
        "delete" : {
          "min_age" : "90m",
          "actions" : {
            "delete" : {
              "delete_searchable_snapshot" : true
            }
          }
        }
      }
    },
    "in_use_by" : {
      "indices" : [
        ".ds-filebeat-data-stream2-2022.05.12-000056",
        ".ds-filebeat-data-stream2-2022.05.12-000055",
        ".ds-filebeat-data-stream2-2022.05.12-000058",
        ".ds-filebeat-data-stream2-2022.05.12-000057",
        ".ds-filebeat-data-stream-2022.05.06-000030",
        ".ds-filebeat-data-stream2-2022.05.12-000059",
        ".ds-filebeat-data-stream-2022.05.06-000031"
      ],
      "data_streams" : [
        "filebeat-data-stream2",
        "filebeat-data-stream"
      ],
      "composable_templates" : [
        "filebeat"
      ]
    }
  }
}

```

And here is the example of one backing index that is stuck in Frozen state:

```auto
    ".ds-filebeat-data-stream2-2022.05.11-000009" : {
      "index" : ".ds-filebeat-data-stream2-2022.05.11-000009",
      "managed" : true,
      "policy" : "filebeat",
      "lifecycle_date_millis" : 1652279563554,
      "age" : "1.26d",
      "phase" : "frozen",
      "phase_time_millis" : 1652284361763,
      "action" : "searchable_snapshot",
      "action_time_millis" : 1652284361763,
      "step" : "wait-for-index-color",
      "step_time_millis" : 1652285568765,
      "repository_name" : "filebeat",
      "snapshot_name" : "2022.05.11-.ds-filebeat-data-stream2-2022.05.11-000009-filebeat-u3zlsfkdt1kl1jm5yvdtca",
      "step_info" : {
        "message" : "index is not green; not all shards are active"
      },
      "phase_execution" : {
        "policy" : "filebeat",
        "phase_definition" : {
          "min_age" : "70m",
          "actions" : {
            "searchable_snapshot" : {
              "snapshot_repository" : "filebeat",
              "force_merge_index" : true
            }
          }
        },
        "version" : 11,
        "modified_date_in_millis" : 1652263549653
      }
    },

```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 16, 2022, 12:30am UTC](https://discuss.elastic.co/t/data-stream-indices-not-moving-from-frozen-to-delete-state/304622/2 "2022-05-16T00:30:21Z")

</div>

> [@elastictor](#):
>
> ```auto
> "message" : "index is not green; not all shards are active"
> 
> ```

Can you check `_cat/inidces/.ds-filebeat-data-stream2-2022.05.11-000009?v`?

---

<div class="post-metadata">

**Author:** ![elastictor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/elastictor/32/105581_2.png) [@elastictor](https://discuss.elastic.co/u/elastictor)\
**Post date:** [June 2, 2022, 12:12pm UTC](https://discuss.elastic.co/t/data-stream-indices-not-moving-from-frozen-to-delete-state/304622/3 "2022-06-02T12:12:56Z")

</div>

Since I am using free version, I will not be able to create searchable snapshots. If I want the indices to be automatically removed, is this not possible? Do I need to manually delete them?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 2, 2022, 9:41pm UTC](https://discuss.elastic.co/t/data-stream-indices-not-moving-from-frozen-to-delete-state/304622/4 "2022-06-02T21:41:43Z")

</div>

Yes, you need to manually remove them if you are using the open source version of Elasticsearch. You can use Curator as well.

---

<div class="post-metadata">

**Author:** ![elastictor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/elastictor/32/105581_2.png) [@elastictor](https://discuss.elastic.co/u/elastictor)\
**Post date:** [June 8, 2022, 7:50am UTC](https://discuss.elastic.co/t/data-stream-indices-not-moving-from-frozen-to-delete-state/304622/5 "2022-06-08T07:50:24Z")

</div>

Recent Elasticsearch versions are not compatible with the latest curator release:

```auto
Elasticsearch version 8.2.2 incompatible with this version of Curator (5.8.4)

```

Is there no other way to automatically delete older indices while using the open source version?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 8, 2022, 11:37pm UTC](https://discuss.elastic.co/t/data-stream-indices-not-moving-from-frozen-to-delete-state/304622/6 "2022-06-08T23:37:18Z")

</div>

8.2 is free, but it not open source.

You can use ILM in 8.2.

---

<div class="post-metadata">

**Author:** ![elastictor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/elastictor/32/105581_2.png) [@elastictor](https://discuss.elastic.co/u/elastictor)\
**Post date:** [June 13, 2022, 6:40am UTC](https://discuss.elastic.co/t/data-stream-indices-not-moving-from-frozen-to-delete-state/304622/7 "2022-06-13T06:40:53Z")

</div>

> 8.2 is free, but it not open source.

Can I still use the 8.2 free version for unlimited usage with the free/base license?  
From where would I get the latest open-source version?

> You can use ILM in 8.2.

If I don't have the enterprise version, to my understanding, I still cannot use searchable snapshots so it will still not be possible to automatically delete indices?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 14, 2022, 12:27am UTC](https://discuss.elastic.co/t/data-stream-indices-not-moving-from-frozen-to-delete-state/304622/8 "2022-06-14T00:27:04Z")

</div>

The last open source version was 7.10.

Searchable Snapshots is a subscription feature, yes. ILM is a free one.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 12, 2022, 12:27am UTC](https://discuss.elastic.co/t/data-stream-indices-not-moving-from-frozen-to-delete-state/304622/9 "2022-07-12T00:27:23Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
