First, thank you for your patience while I attempt to illustrate my issue. I hope I'm being coherent enough.
I am using Apache NiFi to route data into Elasticsearch. Data being ingested into Elasticsearch has indices dynamically created based on
log_type. For example: 'host1_syslog`. I'm only ingesting the current running message log in these instances. There are cases where we need to backfill message data. Doing so creates a monolithic index with historical data added.
What I'd like to do is separate the indices by current message data and historical data. Let say current message data is most recent 30 days. I could have the current message data going to
host1_syslog_old These could be aliased together
host1_syslog but I'm not sure how to handle the index lifecycle policy management for that. The ilm policy just controls the rollover per index. Is there a way to create an ilm policy that would have
host1_syslog_current only contain 30 days worth of data and will rollover data/append older than 30 days to
Am I explaining this well enough?