ILM has rolled over indices way over the limits

Elastic Stack Elasticsearch
1

I setup a policy (ILM) for an index to roll over to a new index when: maximum index size = 50GB or maximum age = 1d

Right now, I have 3 indexes which were created by the policy above, but their size is way above 50GB.

gcp_vpc_flow-000003 green open 4 1 73648356 121.1gb (current index)
shrink-gcp_vpc_flow-000002 green open 1 1 73856698 95.7gb (rolled over)
shrink-gcp_vpc_flow-000001 green open 1 1 73822098 95.8gb (rolled over)

Why are rolled over indexes over 50GB in size?

2

I'm having the same issue (almost). I have the same policy applied but my current index won't rollover at all (unless I manually intervene). Current index is at 80gb. Sorry for jumping in on your question but I think the source of my issue might be similar and as such following it might produce results for me too.

3

The size is for the primary copy, if you have 1 replica, then it will appear to roll over at 100G. If they are over that limit, check Kibana Index Management for ILM errors.

1 Like
4

Agreed with Regenl. I believe the size you see includes both replica and primary shard. I am a little bit confused by the index name though. It says shrink in front. I am not sure if they are indices being rolled over by ILM. Are you sure the roll over working at all?

5

Great, this totally makes sense now. It seems that I have two shards 50GB each: primary and replica, and that's why I've been seeing ~100GB on the shrink indices.

6

No errors in Kibana.

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.