ILM rollover on inconsistent docs count rather than configured configuration

Elastic Stack Elasticsearch
1

I have created ILM policy with following configuration :

{
    "policy": {
        "phases": {
            "hot": {
                "actions": {
                    "rollover": {
                        "max_docs": 15000
                    }
                }
            },
            "delete": {
                "min_age": "1d",
                "actions": {
                    "delete": {}
                }
            }
        }
    }
}

With the index_template for matching indices.
Now when I bootstrap the index with initial index lets say <index-name>-000001 then the expectation is to rollover the index once docs.count reaches to 15000.
The rollover is happening but at random docs.count and I'm not sure why it is happening.

image
image1063×158 29.5 KB

Also when I hit refresh API, then docs count is getting updated at /_cat/indices. Please let me know what is wrong in the config. Did I missing anything?

2

Welcome to our community! :smiley:
Please don't post pictures of text, logs or code. They are difficult to read, impossible to search and replicate (if it's code), and some people may not be even able to see them :slight_smile:

The rollover processes checks the state of the indices every 10 minutes. However there is a fair bit of discrepancy there that doesn't account for that.

What version of Elasticsearch are you on?

3

Thanks for the reply, I'm using elasticsearch-7.16.2. The attached picture is just for reference

4

One finding is when I call /_refresh API manually on the newly created index after predefined interval time, the rollover happening perfectly.
Why I need to call manually the /<index-name-000001>/_refresh API to rollover happens ?

5

The problem was with the refresh_interval. By default, Elasticsearch periodically refreshes indices every second, but only on indices that have received one search request or more in the last 30 seconds.
My indices were not getting search request so couldn't update the indices.
I changed the default setting "refresh_interval":"10s" solved the problem.

Also I changed the _cluster/setting poll interval to 1 minute for testing.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.