Delete all data streams when delete index through data stream API

Elastic Stack Elasticsearch
1

I have index "test-index", and other data streams but "test index" data stream.
When I try to delete this "test-index" index through data stream API (and "test-index" data stream does not exist), I get "acknowledge: true" and all data streams are deleted.

Because of this, we had to solve data loss of all our logs, can you please confirm it's the very unpleasant bug?

Thanks.

2

Could you paste here the exact commands you ran?

Ideally a full reproduction script would help to understand what is happening.

3

This creates some data streams and "test-index" index

for i in `seq  11 20`; do
# Creates a component template for mappings
curl --insecure -X PUT "https://user:pass@localhost:9200/_component_template/my-mappings-${i}?pretty" -H 'Content-Type: application/json' -d"
{
  \"template\": {
    \"mappings\": {
      \"properties\": {
        \"@timestamp\": {
          \"type\": \"date\",
          \"format\": \"date_optional_time||epoch_millis\"
        },
        \"message\": {
          \"type\": \"wildcard\"
        }
      }
    }
  },
  \"_meta\": {
    \"description\": \"Mappings for @timestamp and message fields\",
    \"my-custom-meta-field\": \"More arbitrary metadata\"
  }
}
"
# Creates a component template for index settings
curl --insecure -X PUT "https://user:pass@localhost:9200/_component_template/my-settings-${i}?pretty" -H 'Content-Type: application/json' -d"
{
  \"template\": {
    \"settings\": {
      \"index.lifecycle.name\": \"my-lifecycle-policy-${i}\"
    }
  },
  \"_meta\": {
    \"description\": \"Settings for ILM\",
    \"my-custom-meta-field\": \"More arbitrary metadata\"
  }
}
"
curl --insecure -X PUT "https://user:pass@localhost:9200/_index_template/my-index-template-${i}?pretty" -H 'Content-Type: application/json' -d"
{
  \"index_patterns\": [\"my-data-stream-${i}*\"],
  \"data_stream\": { },
  \"composed_of\": [ \"my-mappings-${i}\", \"my-settings-${i}\" ],
  \"priority\": 500${i},
  \"_meta\": {
    \"description\": \"Template for my time series data\",
    \"my-custom-meta-field\": \"More arbitrary metadata\"
  }
}
"
curl --insecure -X PUT "https://user:pass@localhost:9200/_data_stream/my-data-stream-${i}?pretty"
done

curl --insecure -X PUT "https://user:pass@localhost:9200/test-index

After that I will delete test index with data stream API

curl --insecure -X PUT "https://user:pass@localhost:9200/_data_stream/test-index

And all the data streams are gone.

4

From Delete data stream API | Elasticsearch Guide [7.15] | Elastic (I've added the emphasis);

Deletes one or more data streams and their backing indices

5

Not sure what you are trying to say.

The problem is deleting all other data streams in a cluster, not backing indices.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.