Restoring a single index for a datastream cheatsheet

[seanziee (Kofi)]

I feel like it took me a long time to figure this out and it may be helpful for others. These are the sets of commands that I send when I need to get back an index that already got deleted by ILM but I want to see the data again in my datastream.

The difficulty is that when you restore the index, once ILM realizes that it should be dead because the time has passed, then it auto-deletes it before you can even see it, so you need to send these commands quickly so that ILM doesn't get a chance.

You'll need to find the name of the snapshot, and the name of the .ds index that you want to restore. I'm using the sample of .ds-prod-metricbeat-datastream-2023.09.11-000310 as the expired index, cloud-snapshot-2023.09.28 as the snapshot

Now you want to restore it using

POST _snapshot/found-snapshots/cloud-snapshot-2023.09.28/_restore
{
  "indices": ".ds-prod-metricbeat-datastream-2023.09.11-000310",
  "rename_pattern": "(.+)",
  "rename_replacement": "$1-restored"
  
}

This recreates the index, and appends "-restored"

Now, remove ILM from indices that were deleted due to ilm or else they get created then destroyed automatically

POST .ds-prod-metricbeat-datastream-2023.09.11-000310-restored/_ilm/remove

Now add the index to the datastream

POST _data_stream/_modify
{
  "actions": [
    {
      "add_backing_index": {
        "data_stream": "prod-metricbeat-datastream",
        "index": ".ds-prod-metricbeat-datastream-2023.09.11-000310-restored"
      }
    }
  ]
}

You can check on the status of the restore with

GET _cat/recovery/.ds-prod-metricbeat-datastream-2023.09.11-000310-restored?v&pretty&human

Et Voila! You can now access that data in your visualizations on your dashboard or elsewhere. You can always add a new ILM for that index or just remember to delete it eventually.