Hello, I am looking for the best method to prune indexes with 0 documents. After we migrated to ILM, we have a few indexes which rollover with 0 documents and this can cause an in-balance across a large cluster where several nodes end up with many shards with 0 documents but not much space utilize…

[image] akrzos: We roll over indexes when the shards are approximately 50GiB in size or typically 24 hours old. This is to mainly match previous daily index rollover behavior of Elasticsearch. I would just remove the time based rollover and keep the size one in that case. [image] akrzos: …

[image] warkolm: I would just remove the time based rollover and keep the size one in that case. That effectively makes these no long time-series data, though it would keep the number of 0 doc indexes down to the minimum. It would also permit some time-series data to live much longer than it…

Pruning Indexes with 0 documents

Elastic Stack Elasticsearch

Christian_Dahlqvist (Christian Dahlqvist) August 24, 2020, 6:57pm 2

Why do you end up with empty indices when using rollover? Are you creating too many small specialized indices? Have you set the max age too low?

Topic		Replies	Views
Lifecycle policy min_document to not rollover empty indices Elasticsearch ilm-index-lifecycle-management	1	246	August 30, 2021
Can I delete empty system indices? Elasticsearch	0	58	October 18, 2024
Automatic deletion of read only indices with 0 documents Elasticsearch ilm-index-lifecycle-management	2	666	October 19, 2020
How to delete the docs which are older than 5 hours from an index using curator and the index must remain Elasticsearch curator	8	655	May 13, 2020
ILM keep rolling over empty indexes Elasticsearch ilm-index-lifecycle-management	4	585	June 23, 2023

Pruning Indexes with 0 documents

Related topics