How flexible the ilm rollover is?

Elastic Stack Elasticsearch
1

Hello!

Trying to understand ILM rollover. I created a test on elastic.co and follow the example on:
Tutorial: Automate rollover with ILM | Elasticsearch Guide [8.5] | Elastic

I set rollover on 10 documents. Now, when testing, the rollover never happens on 10 documents added to the Stream. It happens later (on 38 documents, or on 42 documents)...

The question is, how "flexible" is the rule about size, date frame, and document counts? When the ILM Gets triggered? Is there some rule about it?

Here are the ilm/explain:

{
  "indices": {
    ".ds-timeseries-2022.11.30-000001": {
      "index": ".ds-timeseries-2022.11.30-000001",
      "managed": true,
      "policy": "timeseries_policy",
      "index_creation_date_millis": 1669821313078,
      "time_since_index_creation": "6.24m",
      "lifecycle_date_millis": 1669821313078,
      "age": "6.24m",
      "phase": "hot",
      "phase_time_millis": 1669821313304,
      "action": "rollover",
      "action_time_millis": 1669821313504,
      "step": "check-rollover-ready",
      "step_time_millis": 1669821313504,
      "phase_execution": {
        "policy": "timeseries_policy",
        "phase_definition": {
          "min_age": "0ms",
          "actions": {
            "set_priority": {
              "priority": 100
            },
            "rollover": {
              "max_size": "50gb",
              "max_primary_shard_size": "50gb",
              "max_age": "30d",
              "max_docs": 10
            }
          }
        },
        "version": 1,
        "modified_date_in_millis": 1669820971268
      }
    }
  }
}
2

Rollover only checks every few minutes so if you are testing with low volumes test it may be off. It is tuned for real production volumes and for that checking every few minutes gives enough accuracy.

1 Like
3

Christian thanks!

I figure it should be something like that... Something "cron job" like... That explains it! :slight_smile:

Thanks again!

4

Yes, definitely. It is not made for small test cycles and unfortunately we don't do a good job of explaining that.

ILM built for scale on the order of gigabytes, days, hours, etc. And works great for that.

As Christian says it's a background task.

5

Thanks Stephen!

All good. I just asked to confirm my setup is working right!
You guys are doing a great job with Elastic! Keep the good work!

Best regards,

Karl

1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.