ILM how to delete index after days

Elastic Stack Elasticsearch
1

My Enviroment

{
  "name" : "5D776AA",
  "cluster_name" : "rust-cluster",
  "cluster_uuid" : "unlQZ9RcTTy34WDJkHZ1wA",
  "version" : {
    "number" : "7.10.2",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "747e1cc71def077253878a59143c1f785afa92b9",
    "build_date" : "2021-01-13T00:42:12.435326Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

I am using Filebeat and Logstash to read a log file, this create a indices per day with name rust-server-moded (with YYYY.MM.DD), and I made a template for this index

    "index_patterns" : [
      "rust-server-*"
    ],

GET rust-server-moded-2021.02.20/_ilm/explain
Returns

{
  "indices" : {
    "rust-server-moded-2021.02.20" : {
      "index" : "rust-server-moded-2021.02.20",
      "managed" : true,
      "policy" : "rust_server",
      "lifecycle_date_millis" : 1613779377709,
      "age" : "5.6d",
      "phase" : "hot",
      "phase_time_millis" : 1614263069063,
      "action" : "rollover",
      "action_time_millis" : 1613779975765,
      "step" : "check-rollover-ready",
      "step_time_millis" : 1614263069063,
      "is_auto_retryable_error" : true,
      "failed_step_retry_count" : 402,
      "phase_execution" : {
        "policy" : "rust_server",
        "phase_definition" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_size" : "50gb",
              "max_age" : "3d"
            },
            "set_priority" : {
              "priority" : 100
            }
          }
        },
        "version" : 3,
        "modified_date_in_millis" : 1613979144386
      }
    }
  }
}

Shouldn't it be removed after 3 days?

2

If you want to delete the index, you should add a delete action.
Example

PUT _ilm/policy/my_policy
{
  "policy": {
    "phases": {
      "warm": {
        "min_age": "10d",
        "actions": {
          "forcemerge": {
            "max_num_segments": 1
          }
        }
      },
      "delete": {
        "min_age": "30d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}
3

This is what I did:

  1. On dev-tool I pasted your policy with my conditions:
{
  "policy": {
    "phases": {
      "warm": {
        "min_age": "2d",
        "actions": {
          "forcemerge": {
            "max_num_segments": 1
          }
        }
      },
      "delete": {
        "min_age": "3d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

  1. Assign the new policy to my indices. It said me the actual indice has a policy and assign the new will overwrite, I accepted.

  2. GET rust-server-moded-2021.02.19/_ilm/explain return

{
  "indices" : {
    "rust-server-moded-2021.02.19" : {
      "index" : "rust-server-moded-2021.02.19",
      "managed" : false
    }
  }
}
4

If you’ve been using Curator or some other mechanism to manage periodic indices, you have a couple options when migrating to ILM:

  • Set up your index templates to use an ILM policy to manage your new indices. Once ILM is managing your current write index, you can apply an appropriate policy to your old indices.
  • Reindex into an ILM-managed index.
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.