Elastic Cloud AWS: Storage Size Drops after Reboot of entire cluster

Elastic Stack Elasticsearch
1

I am running an instance of Elastic Cloud via the offered AWS facility.
I am saving a large amount of documents to the cluster and starting to reach the max capacity for storage.

When I reach a certain point the indexes will go into READ ONLY mode (which is expected). However, out of curiosity I do a "Full restart. All instances restart simultaneously." When they restart, a significant amount of space becomes available and the indexes are no longer read only.

The document counts in the indexes do not change.

So why does a full restart clear space in the storage?

I dont know if it makes any difference, but I have "index.refresh_interval": "-1" on all my indexes.

Thanks,
Matt

2

It would be useful to compare the outputs of GET /_cat/segments before and after the restart, and to do a similar comparison of GET /_stats?level=shards. While a node is running, some data is kept in multiple places (the translog, and potentially also in more than one segment) but when a node shuts down cleanly it performs a flush which might mean it can discard some of these copies. The API calls above will give a much more detailed picture.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.