Index size is different from ILM index size

Why my index size is different from ilm index size property

health status index                            uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   log-wlb-sysmon-2021.10.01-000014 OaDubXBDTFuwVqYjLIIcXg   1   1   87312747            0    114.4gb         57.2gb
green  open   log-wlb-sysmon-2021.09.30-000013 _ukLur8LR0aY4pUJPnGIcw   1   1   85912426            0    113.9gb         56.9gb
green  open   log-wlb-sysmon-2021.08.16-000015 EtF6odgpQGGRjQ0Nywqtxg   1   1  102154111            0    102.2gb         51.1gb
green  open   log-wlb-sysmon-2021.09.23-000019 tevWknvdRY-XwrEARoNY9g   1   1   57709948        59091     68.4gb         34.2gb
green  open   log-wlb-sysmon-2021.08.27-000017 hAGeCkqhRlCBvmwBLonjaQ   1   1   93303600            0      102gb           51gb
green  open   log-wlb-sysmon-2021.08.19-000016 mkBAnhNMReyHzw0PJwkCHg   1   1   83527835            0      102gb           51gb
green  open   log-wlb-sysmon-2021.09.07-000018 HuDxDct7RGKOcuYFXCKA8w   1   1   92614553            0    101.7gb         50.8gb

Here is my ilm policy for this indices

{
  "winlogbeat_sysmon_policy" : {
    "version" : 14,
    "modified_date" : "2021-10-06T09:06:34.667Z",
    "policy" : {
      "phases" : {
        "hot" : {
          "min_age" : "0ms",
          "actions" : {
            "forcemerge" : {
              "max_num_segments" : 1
            },
            "rollover" : {
              "max_size" : "50gb",
              "max_primary_shard_size" : "50gb",
              "max_age" : "30d"
            },
            "set_priority" : {
              "priority" : 100
            }
          }
        },
        "delete" : {
          "min_age" : "60d",
          "actions" : {
            "delete" : {
              "delete_searchable_snapshot" : true
            }
          }
        }
      }
    }
  }
}

In max size i specified size 50gb and i have one replica for each index so for one index size should be 50gb+50gb(P+R)=100gb.But is different.

Here is something from the docs (emphasis is mine):

ILM runs periodically, checks to see if an index meets policy criteria, and executes whatever steps are needed.

It's controlled by indices.lifecycle.poll_interval, which defaults to 10 minutes. See Index lifecycle management settings in Elasticsearch | Elasticsearch Guide [7.15] | Elastic

So when ILM ran for your index, the size of the index was above 50gb (but not equal to 50gb). That's why you have approximately a bit more than 50gb per shard.

If you want to be closer to he 50gb, you can decrease this value.

i already set this indices.lifecycle.poll_interval: "1m" in cluser settings.

So it indicates that you are getting a lot of documents every minute.

Is there anything wrong with this behavior?

yes,you are right
thanks @dadoonet

That's really not a great idea, you're going to be putting undue pressure on your cluster for very little positive outcome.

@warkolm

But in earlier reply the reason for decreasing this value was i was getting the index size greater than 50gb ,so i decreased this value