Trouble with my ILM

Elastic Stack Elasticsearch
1

Hi all,

I've the following paramters on my ILM :

"policy": {
      "phases": {
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_primary_shard_size": "12gb",
              "max_age": "30m"
            },
            "set_priority": {
              "priority": 100
            }
          }
        },
        "delete": {
          "min_age": "2h",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        },
        "warm": {
          "min_age": "30m",
          "actions": {
            "set_priority": {
              "priority": 50
            }
          }
        }
      }
    },

The settings on the index template is the following :

"settings": {
      "index": {
        "lifecycle": {
          "name": "2h_30mh_90mw"
        },
        "codec": "best_compression",
        "routing": {
          "allocation": {
            "include": {
              "_tier_preference": "data_hot"
            }
          }
        },
        "refresh_interval": "60s",
        "number_of_shards": "1",
        "number_of_replicas": "0"
      }
    }

Anyone could explain to me why it produces index files with a storage size of 20GB ? instead of the 12 GB defined ?

DOn't hesitate to tell me if it's the wrong place to post according to the subject :slight_smile:

Thanks by advance for your reply

Regards,
Christophe

2

Hi @ccaillet,
Can we see the output of _cat/indices/ for the indices in question?

3

ILM tasks are executed on a schedule that per default is every 10 minutes and in the linked documentation you also have this information:

To avoid race conditions, ILM might need to run more than once to execute all of the steps required to complete an action.

So per default it may take 20 minutes for the ILM to complete the action and by this time your index may grow above the configured maximum shard size.

You may change this interval, but keep in mind that this can impact your cluster performance depending on yout lifecycle policies.

I'm not sure ILM works well with short ages and small sizes, it was built to be used thinking on age of days and sizes of a couple of tens GB.

1 Like
4

HI Andrew,

Here is a result on an indice that has the "problem"

green open .ds-cdn-varnish.edge-http-2023.02.22-000064 TtgvEfRsTxSsp_T-1xCBCw 1 0 103568775 0 16gb 16gb

As Leandro said, it seems related to the time between two sequences of ILM schedule. I haven't this behaviour on my production environment which has bigger indices files and of course rollover params are different.

My pre-production cluster has small storage, that's why I have small retention and of course small size for shards and age.

Thanks for your replies.

Regards

5

Nice, glad it got sorted out.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.