Deleting Events From Frozen Data Tier

Elastic Stack Elasticsearch
1

Attempting to delete by query events in a frozen data tier index belonging to a data stream. I've tried targeting the specific index the events are in as well as the datastream name, but I get the following error:

{
        "index": "partial.contoso-2023.10.19-000147",
        "id": "eoERR4sBzZCF6uPiMLfR",
        "cause": {
          "type": "cluster_block_exception",
          "reason": "index [partial.contoso-2023.10.19-000147] blocked by: [FORBIDDEN/8/index write (api)];"
        },
        "status": 403
      }
2

For this to work, you will need to remount the index without write blocked.
This is by design as cold and frozen are meant as archival (read only).

If you really need to remove the data, my approach would be to reindex the data by query (excluding the data you need to remove) and then delete the source index.

3

Thanks @sholzhauer! Mind double-checking my work, will the source index still be picked up properly by the data stream? My query should exclude all documents where the field data_origin is contoso-2023.09.22-000983

POST _reindex?wait_for_completion=false
{
  "source": {
    "index": "partial-.contoso-2023.09.13-000983",
    "query": {
      "bool": {
        "must_not":{
          "match":{
            "data_origin": "contoso-2023.09.22-000983"
          }
        }
      }
    }
  },
  "dest": {
    "index": ".ds.contoso-2023.09.22-000983-reindex",
     "op_type": "create"
  }
}
4

will the source index still be picked up properly by the data stream

If you are referring to the Data View in kibana, then yes it should be picked up.
But it will not be a part of the same datastream.

If you want it to be a part of the same datastream you should replace your dest configuration with the datastream name.

{
    "dest": {
        "index": "contoso",
        "op_type": "create"
    }
}

My query should exclude all documents where the field data_origin is contoso-2023.09.22-000983

Then yes your query looks to be doing precisely that.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.