Hello,
How can i delete the old data from an index, if i create an index everyday i am facing some problems wrt shards performance , so i have only one index and i would like to delete old data. please let me know how can i achieve this
You need to use the delete by query plugin, but it is really inefficient.
Use weekly/monthly and reduce the shard count. That'll be much cheaper than the above.
Using time-based indices does not necessarily mean daily indices. If you have long retention period and reasonably small amounts of data coming, you can instead use monthly indices. With a single index you will need to use the delete by query API, which will be more resource intensive than simply dropping an index.
is there a way to automate this?
i dont prefer using time based indices, i would like to have single index , can you please throw some light on API for clearing the data till last week?
Using a range query based on your timestamp field together with the delete-by-query API should work.
You should really follow @Christian_Dahlqvist and @warkolm advices. Time based indices is the way to go. You will also benefit from index sorting, so faster searches.
any references for deleting old data with powershell which is n days old?
If you are using time based indices, that should be something like:
curl -XDELETE http://localhost:9200/test-2017-06
(let say you have monthly indices).
1 Like
The aforementioned Curator comes with a Windows installer, so you'd be free to use it in power-shell.
I should also point out it has command-line functionality as well (not just YAML-configuration), which should be just as easy to use from powershell.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.