Hello, I am looking for the best method to prune indexes with 0 documents. After we migrated to ILM, we have a few indexes which rollover with 0 documents and this can cause an in-balance across a large cluster where several nodes end up with many shards with 0 documents but not much space utilize…

Why do you end up with empty indices when using rollover? Are you creating too many small specialized indices? Have you set the max age too low?

[image] Christian_Dahlqvist: Why do you end up with empty indices when using rollover? Are you creating too many small specialized indices? Have you set the max age too low? Hi Christian, We have several indexes that have 0 docs so far as we have several "Retention" periods for filebeat logg…

[image] akrzos: We roll over indexes when the shards are approximately 50GiB in size or typically 24 hours old. This is to mainly match previous daily index rollover behavior of Elasticsearch. I would just remove the time based rollover and keep the size one in that case. [image] akrzos: …

[image] warkolm: I would just remove the time based rollover and keep the size one in that case. That effectively makes these no long time-series data, though it would keep the number of 0 doc indexes down to the minimum. It would also permit some time-series data to live much longer than it…

Pruning Indexes with 0 documents

Elastic Stack Elasticsearch

system (system) Closed September 22, 2020, 1:50pm 6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Lifecycle policy min_document to not rollover empty indices Elasticsearch ilm-index-lifecycle-management	1	246	August 30, 2021
Can I delete empty system indices? Elasticsearch	0	58	October 18, 2024
Automatic deletion of read only indices with 0 documents Elasticsearch ilm-index-lifecycle-management	2	666	October 19, 2020
How to delete the docs which are older than 5 hours from an index using curator and the index must remain Elasticsearch curator	8	655	May 13, 2020
ILM keep rolling over empty indexes Elasticsearch ilm-index-lifecycle-management	4	585	June 23, 2023

Pruning Indexes with 0 documents

Related topics