Index lifecycle policy not applied to active index

Elastic Stack Elasticsearch
1

Hello,

I have created a new ELK 7.0 stack and am struggling with index roll over.
Currently I have a packetbeat-7.0.0-2019.04.21-000001 index that is 55gb large, the alias of that index is packetbeat-7.0.0.
The attached lifecycle policy is more or less the basis policy, 50gigabytes but max age of 6 hours
Below is the json output of the policy

{
    "policy": {
        "phases": {
            "hot": {
                "min_age": "0ms",
                "actions": {
                    "rollover": {
                        "max_age": "6h",
                        "max_size": "50gb"
                    }
                }
            }
        }
    }
}

When I check the phase definition on the index it gives the following output

{
  "policy": "packetbeat-7.0.0",
  "phase_definition": {
    "min_age": "0ms",
    "actions": {
      "rollover": {
        "max_size": "50gb",
        "max_age": "30d"
      }
    }
  },
  "version": 1,
  "modified_date_in_millis": 1555844316520
}

GET packetbeat-7.0.0-2019.04.21-000001/_ilm/explain gives the following output

{
  "indices" : {
    "packetbeat-7.0.0-2019.04.21-000001" : {
      "index" : "packetbeat-7.0.0-2019.04.21-000001",
      "managed" : true,
      "policy" : "packetbeat-7.0.0",
      "lifecycle_date_millis" : 1555844317187,
      "phase" : "hot",
      "phase_time_millis" : 1555844317507,
      "action" : "rollover",
      "action_time_millis" : 1555844530404,
      "step" : "check-rollover-ready",
      "step_time_millis" : 1555844530404,
      "phase_execution" : {
        "policy" : "packetbeat-7.0.0",
        "phase_definition" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_size" : "50gb",
              "max_age" : "30d"
            }
          }
        },
        "version" : 1,
        "modified_date_in_millis" : 1555844316520
      }
    }
  }
}

I was expecting a rollover based on the data size of 50gb but nothing happened.
What should I change or do to start the automatic roll over to a new index after eq. 6 hours?
thnxz
John

2

Hmmm, can it be that the size being shown is 55gb while the primaries is "only" 28gb and not near the 50gb limit defined in the policy?
GET packetbeat-7.0.0-2019.04.21-000001/_stats

{
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "failed" : 0
  },
  "_all" : {
    "primaries" : {
      "docs" : {
        "count" : 81593179,
        "deleted" : 0
      },
      "store" : {
        "size_in_bytes" : 30066485090
      },
      "indexing" : {
        "index_total" : 72882006,
        "index_time_in_millis" : 53494651,
        "index_current" : 0,
        "index_failed" : 0,
        "delete_total" : 0,
        "delete_time_in_millis" : 0,
        "delete_current" : 0,
        "noop_update_total" : 0,
        "is_throttled" : false,
        "throttle_time_in_millis" : 0
      },
1 Like
3

Fixed it.
I removed the lifecycle policy from the index and added it again.
Now it is working as expected

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.