I want to delete documents older than 3 months in my index. And I have read the doc here about using ILM to create time-based indices and apply ILM policies. However, this doesn't seem to fit our use case. We're using id we generate to write/update/read documents in the index. If we're to create time-based indices, we want to keep documents with a specific id unique across indices and this doesn't seem possible from what I learned.
Is there any other ways to automatically delete old documents that will fit my use case ?
Right now we have data older than 3 months. But we only need data within 3 months. For now we can do a manual delete-by-query, but in the future we want to automatically delete those old data. Is that possible ?
If that is the case time based indices may not be right for your use case and you will need to implement a way to trigger a periodic delete by query yourself as Elasticsearch does not support this.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.