I have an Elastic cloud cluster where I have 2 instances. Each instance is 120GB in size and works as replicas in different Availability Zones for redundancy.
Something strange happened that I can't understand what it could be:
Instance 4 started consuming twice the size of instance 3 and I didn't change any cluster properties.
With this exaggerated consumption (doubled) I had an unavailability of access to my cluster because the instance that was twice in size practically reached the space limit and it had the indexes that control authentication in my cluster.
I did take a look at the _cat/indices it looks pretty normal EXCEPT
You have a lot of unassigned shards because you have a Warm tier with just 1 node and your indices have a replica so there is no node to put the replicas on. 1st I would add another warm node and let it balance / assigned the missing shards and take a look. Or if you truly only want 1 primary ... i.e. not resilience in the warm you need to set the Replicas to 0 and also do that in the ILM, I would not recommend this... best to just add another small warm node.
Then I would suggest Opening a Support Ticket... something is not right
What concerns me is that a 4GB i3 should not have 204 GB of Disk it should just have 120.
I would put all this information in the support ticket.
You are right to be weird about the storage size.
Elastic support increased the size of instance 4 yesterday, because it reached the maximum configuration of this instance, which was 120GB of space.
Now, the point that I find super strange is that on node 3 it allocates only 60GB and on node 4 (hot replica) it allocates twice as much.
How does this relate to the warm layer I left without redundancy (only 1 node)?