Documents deleted from all indices

[venku]

I am running a single ElasticSearch node for past week and half and my ILM policy was to delete the index if it is 10 days old after the rollover. I have a day or 5GB limit to move from hot to delete phase. However when I checked the indices today I see that the docs.count is showing zeros for all the indices except the latest one. Can someone help me what could have happened here. Where should I look to debug this issue of documents getting deleted?

ILM Policy:

{
  "policy": {
"phases": {
  "hot": {
    "min_age": "0ms",
    "actions": {
      "rollover": {
        "max_age": "1d",
        "max_size": "5gb"
      },
      "set_priority": {
        "priority": 10
      }
    }
  },
"delete": {
    "min_age": "10d",
    "actions": {
      "delete": {}
    }
  }
}
  }
}

[Christian_Dahlqvist]

My guess would be that you are writing directly to the index with data instead of the write alias that you should use with rollover. What does your output config look like in beats/Logstash?

[venku]

You are absolutely correct!! I went back and looked at logstash output and I made a mistake for this index. I am using the index name instead of the alias. Do I have to remove all the indices before I correct this and restart logstash?

[Christian_Dahlqvist]

I do not think you need to do anything after the update, assuming Logstash is set to automatically reload config periodically, but check that data starts flowing into the latest index.

[venku]

I am using logstash docker and will check further if it reload from the volume automatically.
I want to ask you one last thing, how did guess it was using index name instead of alias? I mean why using index name deleted the docs after some days?

[Christian_Dahlqvist]

No data was deleted from the empty indices as none was ever written to them. All data continued to be written to the initial index with the lowest sequence number and rollover rolled over empty indices based on time alone.

If you query the date range covered by data in the first index I expect you will see that it covers the entire period.

[venku]

I see. Thank you for all the help @Christian_Dahlqvist!!