Index not moving to delete phase

Hello,
I have Elasticsearch 8 on K8s. Fluentbit sends logs to ES8. Everyday new indexes are created based on date; likes this:
backend-app-2023.05.09
backend-app-2023.05.10
backend-app-2023.05.11
...
I would like enable ILM on these indexes. Regarding to the documentation, I prepared following configuration:

PUT _ilm/policy/delete-policy
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "30s",
        "actions": {
          "set_priority": {
            "priority": 100
          },
          "rollover": {
            "max_primary_shard_size": "1MB"
          }
        }
      },
      "delete": {
        "min_age": "3m",
        "actions": {
          "delete": {
            "delete_searchable_snapshot": true
          }
        }
      }
    }
  }
}

PUT _index_template/short-live
{
  "index_patterns": [
    "backend-*"
  ],
  "template": {
    "settings": {
      "number_of_shards": 1,
      "number_of_replicas": 0,
      "index.lifecycle.name": "delete-policy",
      "index.lifecycle.rollover_alias": "short-live"
    },
    "aliases" : {
        "short-live" : {}
    }
  },
  "priority": 500
}

PUT backend-000001
{
  "aliases": {
    "short-live": {
      "is_write_index": true
    }
  }
}

My configuration doesn't work. Nothing is deleted. As far as I understood, the index not moving to delete phase. The existing index size keeps increase.

Could you please advise?

Thanks & Regards

You don't want this, you need to change fluentbit to write to short-live and let ILM create the underlying indices.

if I use "short-live" as index name, the alias "short-live" cannot be used in index-template.

Regarding to your advise, If I write fluent-bit logs to "short-live", ES doesn't let me create backend-000001 index:

{
  "error": {
    "root_cause": [
      {
        "type": "invalid_alias_name_exception",
        "reason": "Invalid alias name [short-live]: an index or data stream exists with the same name as the alias"
      }
    ],
    "type": "invalid_alias_name_exception",
    "reason": "Invalid alias name [short-live]: an index or data stream exists with the same name as the alias"
  },
  "status": 400
}

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.