I would what to know how to rotate old records from ES index, coz when i
delete documents with delete query API it does not free up disk space. Eg.
i want to remove records older than 3 days in my index. For me disk space
is an issue.
When you run delete by query, documents aren't immediately deleted. They
are marked as deleted and the data gets removed in the process of merging,
which runs asynchronously in the background. There are some settings that
make merging more "sensitive" to deletes, have a look here:
Alternatively, you can force a merge by running optimize:
But this is also problematic because it will thrash your I/O and invalidate
much of your caches.
Usually, a better option if you have expiring data is to have time-based
indices (an index per day, for example). You can then use something like
the Elasticsearch Curator to expire entire indices as the days go by:
Here's a nice blog post about it:
Using time-based indices is typically better because removing entire
indices is super-fast (really remove the associated files and little
metadata) and doesn't mess up with the rest of your data and caches.
I would what to know how to rotate old records from ES index, coz when i
delete documents with delete query API it does not free up disk space. Eg.
i want to remove records older than 3 days in my index. For me disk space
is an issue.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.