# Documents deleted from all indices

**URL:** <https://discuss.elastic.co/t/documents-deleted-from-all-indices/253900>\
**Category:** Elasticsearch\
**Created:** [October 31, 2020, 1:40pm UTC](https://discuss.elastic.co/t/documents-deleted-from-all-indices/253900 "2020-10-31T13:40:10Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![venku](https://avatars.discourse-cdn.com/v4/letter/v/a698b9/32.png) [@venku](https://discuss.elastic.co/u/venku)\
**Post date:** [October 31, 2020, 1:40pm UTC](https://discuss.elastic.co/t/documents-deleted-from-all-indices/253900/1 "2020-10-31T13:40:11Z")

</div>

I am running a single ElasticSearch node for past week and half and my `ILM` policy was to delete the index if it is 10 days old after the rollover. I have a day or 5GB limit to move from `hot` to `delete` phase. However when I checked the indices today I see that the `docs.count` is showing zeros for all the indices except the latest one. Can someone help me what could have happened here. Where should I look to debug this issue of documents getting deleted?

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/f/df18a88770adeb2d0e95055536f3c2df5d16aa89.png)

**ILM Policy:**

```
{
  "policy": {
"phases": {
  "hot": {
    "min_age": "0ms",
    "actions": {
      "rollover": {
        "max_age": "1d",
        "max_size": "5gb"
      },
      "set_priority": {
        "priority": 10
      }
    }
  },
"delete": {
    "min_age": "10d",
    "actions": {
      "delete": {}
    }
  }
}
  }
}
```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 31, 2020, 3:18pm UTC](https://discuss.elastic.co/t/documents-deleted-from-all-indices/253900/2 "2020-10-31T15:18:26Z")

</div>

My guess would be that you are writing directly to the index with data instead of the write alias that you should use with rollover. What does your output config look like in beats/Logstash?

---

<div class="post-metadata">

**Author:** ![venku](https://avatars.discourse-cdn.com/v4/letter/v/a698b9/32.png) [@venku](https://discuss.elastic.co/u/venku)\
**Post date:** [October 31, 2020, 3:57pm UTC](https://discuss.elastic.co/t/documents-deleted-from-all-indices/253900/3 "2020-10-31T15:57:43Z")

</div>

You are absolutely correct!! I went back and looked at logstash output and I made a mistake for this index. I am using the index name instead of the alias. Do I have to remove all the indices before I correct this and restart logstash?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 31, 2020, 4:00pm UTC](https://discuss.elastic.co/t/documents-deleted-from-all-indices/253900/4 "2020-10-31T16:00:36Z")

</div>

I do not think you need to do anything after the update, assuming Logstash is set to automatically reload config periodically, but check that data starts flowing into the latest index.

---

<div class="post-metadata">

**Author:** ![venku](https://avatars.discourse-cdn.com/v4/letter/v/a698b9/32.png) [@venku](https://discuss.elastic.co/u/venku)\
**Post date:** [October 31, 2020, 4:05pm UTC](https://discuss.elastic.co/t/documents-deleted-from-all-indices/253900/5 "2020-10-31T16:05:20Z")

</div>

I am using logstash docker and will check further if it reload from the volume automatically.  
I want to ask you one last thing, how did guess it was using index name instead of alias? I mean why using index name deleted the docs after some days?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 31, 2020, 4:15pm UTC](https://discuss.elastic.co/t/documents-deleted-from-all-indices/253900/6 "2020-10-31T16:15:10Z")

</div>

No data was deleted from the empty indices as none was ever written to them. All data continued to be written to the initial index with the lowest sequence number and rollover rolled over empty indices based on time alone.

If you query the date range covered by data in the first index I expect you will see that it covers the entire period.

---

<div class="post-metadata">

**Author:** ![venku](https://avatars.discourse-cdn.com/v4/letter/v/a698b9/32.png) [@venku](https://discuss.elastic.co/u/venku)\
**Post date:** [October 31, 2020, 4:22pm UTC](https://discuss.elastic.co/t/documents-deleted-from-all-indices/253900/7 "2020-10-31T16:22:55Z")

</div>

I see. Thank you for all the help @Christian_Dahlqvist!!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 28, 2020, 4:23pm UTC](https://discuss.elastic.co/t/documents-deleted-from-all-indices/253900/8 "2020-11-28T16:23:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.