Transform destination index rollover

Elastic Stack Elasticsearch
, ,
1

Hello,

It has been a while (I think) since this question came up on here, so I thought I'd check whether the answer had changed or not.

I have a transform which produces consolidated data with a time-series element to it.

This data is currently being written out to one index, but since the volume of the data is significant and the retention requirement lengthy, I would like to roll the index over from time to time.

Ideally, I'd like to use a datastream to handle this but, historically, this has not been possible.

The next-best alternative answer I could find was to use an index with an ingest pipeline which uses the date_index_name processor (as documented here) and a standard ILM policy on the destination indices to handle tiering, etc.

Is the date_index_name method still the recommended method?

Many thanks!

2

so I thought I'd check whether the answer had changed or not.

Unfortunately not, the data stream is still not allowed as a destination index.
See documentation: Transform limitations | Elasticsearch Guide [8.11] | Elastic

So if you have an ingest-pipeline workaround that is working for you then that is the way to go.

3

Thanks for the reply, and the confirmation.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.