No way to rollover mutable timeseries data

Since datastreams are append-only, Tutorial: Automate rollover with ILM | Elasticsearch Guide [8.10] | Elastic suggests an alternative technique for rolling over mutable documents:

In these cases, you can use an index alias to manage indices containing the time series data and periodically roll over to a new index.

I'm confused about how this could work. As I understand it, rolled over aliases only support writes to the most recent index, which means all existing data would cease to be mutable whenever a rollover occurs. It doesn't seem like it would ever be safe to rollover a live application, even if it only needs to update the most recent documents.

The only way I've found to manage the retention of documents that have mutable fields is to require clients to append the date onto the index name. While we can still apply an ILM policy via an index template, it will lack the benefits of configurable rollover (particularly the ability to specify a target index size).

It's a shame this common case is not better supported by data streams and aliases. I also feel the documentation is misleading and should not suggest a solution that has such a crucial flaw.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.