I am using Logstash to ingest log data to Elasticsearch, Indices are created daily, that every day's index has a unique name like
index_name-YYYY-MM-DD . So, after 30 days, there will be 30 indices. On day 31st, I want the index made at day 1 to get deleted, and on 32nd day, I want 2nd day's index to get deleted.
I had a look at Index Lifecycle Policies. It is showing option through curl but is there any way to do through logtash conf file?
This is what ILM was designed for. It can not be done through Logstash.
is there any other way to automate the deletion
Why do you not want to use ILM? It is what I would recommend.
You could also use Curator, which runs as an external process. This is the old way of managing index lifecycles that ILM is replacing.
I tried using ILM my data is stucked at hot only it does not move to other section
Then I would recommend opening a separate thread to solve that issue rather than dismiss ILM. When you do so, make sure you share the full policy as well as node configurations and version information.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.