Lifecycle Management rotate on a daily basis

Elastic Stack Elasticsearch
1

How do i correctly config ILM to rotate my index on a daily basis and delete the index after 120 days?

I tried this config, but it doesn't rotated the index:

"logstash" : {
    "version" : 1,
    "modified_date" : "2019-03-27T11:42:32.848Z",
    "policy" : {
      "phases" : {
        "warm" : {
          "min_age" : "14d",
          "actions" : {
            "allocate" : {
              "number_of_replicas" : 0,
              "include" : { },
              "exclude" : { },
              "require" : { }
            }
          }
        },
        "hot" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_size" : "50gb",
              "max_age" : "1d"
            }
          }
        },
        "delete" : {
          "min_age" : "120d",
          "actions" : {
            "delete" : { }
          }
        }
      }
    }
  }
2

Hi Gruselglatz,

The policy looks correct to me, can you use the ILM Explain API on the index to see what's going on, should be:

GET /<index-name>/_ilm/explain?pretty&human

This will give us an idea of where the policy is and whether it's hit any errors while executing.

3

Hi dakrone!

it looks like its rotating now, but with one day delay.
I get this now:

Real Date -> index name
28.03.2019 -> logstash-2019.03.27-000001
29.03.2019 -> logstash-2019.03.28-000002
30.03.2019 -> logstash-2019.03.29-000003
31.03.2019 -> logstash-2019.03.30-000004
01.04.2019 -> logstash-2019.03.31-000005

but what i want to get is:
Real Date -> index name
28.03.2019 -> logstash-2019.03.28-000001
29.03.2019 -> logstash-2019.03.29-000001
30.03.2019 -> logstash-2019.03.30-000001
31.03.2019 -> logstash-2019.03.31-000001
01.04.2019 -> logstash-2019.04.01-000001

And when i get over xx million docs the index should rotate on the same day and increment the last number.

the output of the API-call:

GET /logstash-2019.03.31-000005/_ilm/explain?pretty&human


{
  "indices" : {
    "logstash-2019.03.31-000005" : {
      "index" : "logstash-2019.03.31-000005",
      "managed" : true,
      "policy" : "logstash",
      "lifecycle_date" : "2019-03-31T13:19:44.376Z",
      "lifecycle_date_millis" : 1554038384376,
      "phase" : "hot",
      "phase_time" : "2019-03-31T13:19:44.477Z",
      "phase_time_millis" : 1554038384477,
      "action" : "rollover",
      "action_time" : "2019-03-31T13:29:44.419Z",
      "action_time_millis" : 1554038984419,
      "step" : "check-rollover-ready",
      "step_time" : "2019-03-31T13:29:44.419Z",
      "step_time_millis" : 1554038984419,
      "phase_execution" : {
        "policy" : "logstash",
        "phase_definition" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_size" : "50gb",
              "max_age" : "1d"
            }
          }
        },
        "version" : 1,
        "modified_date" : "2019-03-27T11:42:32.848Z",
        "modified_date_in_millis" : 1553686952848
      }
    }
  }
}
4

@dakrone do you know something new? I think it should be fixable by configuring the rotation time to midnight every day and not 1PM afternoon.

how could I achieve it?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.