I manage an elasticsearch cluster with about 25 different applications sending their logs to it. The naming convention for the index logging is this, apps-logs--yyyy.MM.dd.
We have apps that do load testing and during this testing write an enormous amount of data. A few days after the load test that data is not needed and can be deleted. With our current Indexing it's not that easy to figure out what needs to be deleted. I would like to have an automated way to purge the load test data from the normal data. The normal data we keep for 60 days.
My idea is to use an index alias that the applications can send their logs too.
New Alias "alias-appname-yyyy.MM.dd" -> apps-logs--yyyy.MM.dd. Then during a load test update the alias to point to test-logs--yyyy.MM.dd.
Then I have a job that runs and deletes test-* if it's older than a week.
How do I create an alias with the date stamp like I described, that is different every day? I would like to create the alias once, and not every day.
Is there a better way to handle the use case I described ?