Before writing this post, I started by gathering available information on this platform, official documentation, related articles.
Examples to reindex basic indices are easy to find but when it comes to datastream, then the way to proceed is not clearly defined.
Here is my context:
I started a datastream one month ago and it appears yesterday, it entered the rollover phase producing a new index. As expected my new indice is using a newer version of mapping from the index template .. resulting in conflicts within my complete datastream, partially breaking charts, dashboards.
Then I searched for recommendations on how to properly reindex a datastream .. here are the following options:
- Reindex the complete datastream into a new one (ensuring the new one matches the expected index template)
- Reindex rollover indices one per one to a new datastream
Both are basically ending with the same outcome but what if I need to guarantee no downtime when I have GBs to recover ?
Let's assume, DS-01 is my current datastream and DS-02 my new one.
At first alias DS-Generic points to DS-01
During such major reindexation, shall I :
- trigger a rollup on DS-01 to create a (N+1)th new indice
- reindex DS-01 from indice 1 to N where new events will be sent to the newly created N+1 th indice
- link alias DS-Generic to DS-02 (write enabled)
- reindex DS-01 N+1 into DS-02
Or can I simply link DS-Generic to DS-02 so new events will be handled by DS-02 and perform reindexation at the same time ?
As far as i understood, datastreams are appending events to the last indice, so with the second approach I fear order in timestamp will not be preserved resulting in DS-02 having mixed timestamps within the same indice.
In other words, are there any good practices to perform a "clean" reindexation on an active datastream not loosing data ?
Ps: this is my first post on the platform... feel free to ask me rephrasing some parts if needed.
Thanks for any support and I hope this thread will be helpful to anyone struggling with reindexing datastreams.