Indices is not deleting after 60 days of retention

Elastic Stack Elasticsearch
1

Using ELK 7.11
Following are the indicies that should be deleted

health status index                         uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   log-pb-flow-2021.04.05-000001 _HTx3o3iTEW25cP-anki-A   1   1   37767666            0     39.2gb         19.6gb
green  open   log-pb-flow-2021.06.04-000003 McUrYPO5S6WjZmwoL_gj2g   1   1    1780730            0      2.1gb            1gb
green  open   log-pb-flow-2021.05.05-000002 CkJzzRBxQ1m1BDD3Rkqcjg   1   1   35851127            0     37.4gb         18.7gb

log-pb-flow-2021.04.05-000001 should be deleted on june 5 but it didn't
GET log-pb-flow-2021.04.05-000001/_ilm/explain

{
  "indices" : {
    "log-pb-flow-2021.04.05-000001" : {
      "index" : "log-pb-flow-2021.04.05-000001",
      "managed" : true,
      "policy" : "packetbeat_flow_policy",
      "lifecycle_date_millis" : 1620217537126,
      "age" : "32.77d",
      "phase" : "hot",
      "phase_time_millis" : 1617625511448,
      "action" : "complete",
      "action_time_millis" : 1620217538464,
      "step" : "complete",
      "step_time_millis" : 1620217538464,
      "phase_execution" : {
        "policy" : "packetbeat_flow_policy",
        "phase_definition" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_size" : "50gb",
              "max_age" : "30d"
            },
            "set_priority" : {
              "priority" : 100
            }
          }
        },
        "version" : 5,
        "modified_date_in_millis" : 1616673431501
      }
    }
  }
}

GET log-pb-flow-2021.04.05-000001/_settings

{
  "log-pb-flow-2021.04.05-000001" : {
    "settings" : {
      "index" : {
        "lifecycle" : {
          "name" : "packetbeat_flow_policy",
          "rollover_alias" : "log-pb-flow",
          "indexing_complete" : "true"
        },
        "routing" : {
          "allocation" : {
            "include" : {
              "_tier_preference" : "data_content"
            }
          }
        },
        "refresh_interval" : "1s",
        "number_of_shards" : "1",
        "provided_name" : "<log-pb-flow-{now/d}-000001>",
        "creation_date" : "1617625511104",
        "priority" : "100",
        "number_of_replicas" : "1",
        "uuid" : "_HTx3o3iTEW25cP-anki-A",
        "version" : {
          "created" : "7110299"
        }
      }
    }
  }

ILM Policy for this index
GET _ilm/policy/packetbeat_flow_policy

{
  "packetbeat_flow_policy" : {
    "version" : 5,
    "modified_date" : "2021-03-25T11:57:11.501Z",
    "policy" : {
      "phases" : {
        "hot" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_size" : "50gb",
              "max_age" : "30d"
            },
            "set_priority" : {
              "priority" : 100
            }
          }
        },
        "delete" : {
          "min_age" : "60d",
          "actions" : {
            "delete" : {
              "delete_searchable_snapshot" : true
            }
          }
        }
      }
    }
  }
}

I have other indices also, which is also not deleting

2

If I remember correctly the next phase time starts ticking at the rime of rollover. As each index covers up to 30 days I believe you have actually configured a total retention period of up to 90 days.

3

so i have to set 30 days in delete policy so that it become 60 days

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.