We are using a Ruby wrapper library called Searchkick which provides great functionality for searching within our application environment. However it is not great with managing Timeseries data.
Our objective is to be able to use ILM and a Hot/Cold architecture to better manage resources. Currently the library will write data into a single index and it will just grow forever. We have a temporary solution of running a
_delete_by_query to purge records within the given index older than 30 days.
The library code executes searches against a single named index:
foo_production_<timestamp>. The only time this index name would change is if for some reason we needed to reindex all of the data from source.(this does not happen often)
My question is, is there a way to implement a rollover strategy without making application/library code changes?
Are we able to implement an ILM policy such that the original index name always remains writeable, and we can maintain the ability to search across the newly created index. In other words can we create an alias with the same name as an index? Does it make sense to convert to a datastream? How would that impact searching?
If we can not get around making some application level change, is there documentation on how to implement the client side logic of index creation to support ILM on the server?