Transform destination index rollover


It has been a while (I think) since this question came up on here, so I thought I'd check whether the answer had changed or not.

I have a transform which produces consolidated data with a time-series element to it.

This data is currently being written out to one index, but since the volume of the data is significant and the retention requirement lengthy, I would like to roll the index over from time to time.

Ideally, I'd like to use a datastream to handle this but, historically, this has not been possible.

The next-best alternative answer I could find was to use an index with an ingest pipeline which uses the date_index_name processor (as documented here) and a standard ILM policy on the destination indices to handle tiering, etc.

Is the date_index_name method still the recommended method?

Many thanks!

so I thought I'd check whether the answer had changed or not.

Unfortunately not, the data stream is still not allowed as a destination index.
See documentation: Transform limitations | Elasticsearch Guide [8.11] | Elastic

So if you have an ingest-pipeline workaround that is working for you then that is the way to go.

Thanks for the reply, and the confirmation.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.