# .ds-logstash shards remain

**URL:** <https://discuss.elastic.co/t/ds-logstash-shards-remain/316734>\
**Category:** Elasticsearch\
**Tags:** ilm-index-lifecycle-management\
**Created:** [October 17, 2022, 6:50am UTC](https://discuss.elastic.co/t/ds-logstash-shards-remain/316734 "2022-10-17T06:50:41Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![waleedkh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/waleedkh/32/112125_2.png) [@waleedkh](https://discuss.elastic.co/u/waleedkh)\
**Post date:** [October 17, 2022, 6:50am UTC](https://discuss.elastic.co/t/ds-logstash-shards-remain/316734/1 "2022-10-17T06:50:41Z")

</div>

I have an issue in that I have 2200 of these .ds-logstash-DATE indices that are not being cleaned out as part of the ILM policy. They are from a data stream.  
Below is the linked ILM policy

```auto
PUT _ilm/policy/logstash-lifecycle-policy
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_primary_shard_size": "25gb",
            "max_age": "1d"
          },
          "set_priority": {
            "priority": 100
          }
        }
      },
      "delete": {
        "min_age": "14d",
        "actions": {
          "delete": {
            "delete_searchable_snapshot": true
          }
        }
      }
    }
  }
}
The data stream is also part of the same ILM policy.

Any ideas what these are and why they remain? They are causing my shard count to reach it's maximum which is then causing errors.

Tx

```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [October 17, 2022, 11:47pm UTC](https://discuss.elastic.co/t/ds-logstash-shards-remain/316734/2 "2022-10-17T23:47:35Z")

</div>

Can you share an [explain](https://www.elastic.co/guide/en/elasticsearch/reference/current/ilm-explain-lifecycle.html) on one of them?

---

<div class="post-metadata">

**Author:** ![waleedkh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/waleedkh/32/112125_2.png) [@waleedkh](https://discuss.elastic.co/u/waleedkh)\
**Post date:** [October 19, 2022, 4:46am UTC](https://discuss.elastic.co/t/ds-logstash-shards-remain/316734/3 "2022-10-19T04:46:14Z")

</div>

Please see below:

```auto
{
  ".ds-logstash-2022.09.30-2022.10.10-000013" : {
    "aliases" : { },
    "mappings" : {
      "_data_stream_timestamp" : {
        "enabled" : true
      },
      "properties" : {
        "@timestamp" : {
          "type" : "date"
        }
      }
    },
    "settings" : {
      "index" : {
        "lifecycle" : {
          "name" : "logstash-lifecycle-policy",
          "rollover_alias" : "logstash",
          "indexing_complete" : "true"
        },
        "routing" : {
          "allocation" : {
            "include" : {
              "_tier_preference" : "data_hot"
            }
          }
        },
        "hidden" : "true",
        "number_of_shards" : "3",
        "provided_name" : ".ds-logstash-2022.09.30-2022.10.10-000013",
        "creation_date" : "1665431888858",
        "priority" : "100",
        "number_of_replicas" : "1",
        "uuid" : "xxxxxxxxx",
        "version" : {
          "created" : "7170299"
        }
      }
    },
    "data_stream" : "logstash-2022.09.30"
  }
}

```

---

<div class="post-metadata">

**Author:** ![waleedkh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/waleedkh/32/112125_2.png) [@waleedkh](https://discuss.elastic.co/u/waleedkh)\
**Post date:** [October 24, 2022, 6:56am UTC](https://discuss.elastic.co/t/ds-logstash-shards-remain/316734/4 "2022-10-24T06:56:11Z")

</div>

Hi just checking if anyone has a possible solution here?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 21, 2022, 6:56am UTC](https://discuss.elastic.co/t/ds-logstash-shards-remain/316734/5 "2022-11-21T06:56:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.