How to find an index that was rolled over

Elastic Stack Kibana
1

Hello, I am trying to query an old index but I can't seem to find it. The index I am looking for is " .ds-logs-system.security-default-2024.04.15-000010" which I have found a record of in the ILM history index. The record indicates that the index was rolled over and that it completed the rollover. Which from my understanding means that the index can still be queried, it just can't be written to.

image
image1209×625 39.4 KB

This is the new index that was created but it does not contain the data I am looking for. The search should have pulled up the older index because I haven't configured any aliases for that index but it's only pulling up the new one. This is the query I used

GET _cat/indices/.ds-logs-system.security*?v

image

The ILM policy applying to that index has the delete phase set at 365 days.

image
image2292×904 141 KB

Just to confirm the results from the API here are the search results from the GUI.

image
image2322×535 41.2 KB

Anyone have any idea where the index went? Am I not searching for it correctly?

Thanks for reading!

2

Bumping for visibility, I think sufficient time has passed. Could anyone point me in the right direction please?

3

If running GET _cat/indices/ does not show the index, then it was deleted.

It was deleted manually or it is not using the policy you think it is using, but if the indice does not show up when using _cat/indices it means that the index does not exist anymore in your cluster.

Can you go into Index Management > Data Streams, look for logs-system.security-default data stream and share the pop-over information side panel that will open?

4

Hi, thanks for getting back to me.

Here is the info on the data stream:

image

Clicking the linked lifecycle policy shows that it should be deleted after 365 days and the applied index template confirms that.

image
image2289×1157 183 KB

image

Maybe it was deleted manually, although I don't remember doing that myself. It's possible I did it accidentally while trying to fix something else. Is there a way to confirm if/when that index was deleted?

5

Not anymore, you would need to have the audit logs enabled before, and the audit logs also requires a paid license, not sure if you have as you didn't mention.

From what you shared, your policy would have not removed it, so it was deleted manually.

1 Like
6

Unfortunately I don't. Well nothing I can do about it I guess except be more careful. Thank you for your time.

7

So I finally found what the issue was. The hot, warm and cold phases were all set at 30 days and the delete phase at 365 days. BUT. The cold phase had the option enabled that says "Delete data after this phase". So after 30 days the data was getting deleted.

image
image2271×514 40.2 KB

Silly mistake, I honestly can't remember setting it up that way but it makes sense that that was the issue. Will have to wait a while for the indexes to start rolling over to confirm that that was the issue but I'm pretty confident that's what it was.