We are using ES as a data store for events from devices. By the end of the year I expect to have a few 100 million events being written every day.
My plan is to create two aliases which will get used by clients:
- An alias ("events-current") that points to the current day's index
- Another ("events-all") that contains all of the event indices.
To do this I am planning to create a script that will:
- Export the mappings from the index behind events-current
- Create a daily index "events-YYYY.MM.DD"
- Apply the mappings from the previous day's index to the new index
- Moving removing the previous day's index from the "events-current" alias
- Adding the new index to both the "events-current and "events-all" aliases
I can do this with shell scripts but there has to be a better way. I'm pretty sure I'm doing the same thing that logstash does by default but wanted to know if I'm missing anything or anyone had a suggestion of a better way to set this up.