I wanted to discuss the possibility of an implementation with the help of ILM policies.
I am currently storing application metrices (flat JSON) into an Elasticsearch index. Only the records that have been indexed in the last 15 days from today are important to us. Data (records) older than 15 days should be moved to cold storage as search requirements for older records are very limited.
I tried implementing the above requirement with the help of an ILM policy of 15d age. But this policy moves the entire index to cold storage rather than just the records that are older than 15 days.
Any suggestions on how I can implement this requirement via ILM policies?