How to switch dataview/lens to new index without downtime?

[defalt]

Hello ,

we have the following setup:

• Elastic 8.5.1 Cluster, Kibana 8.5.1.
• Multiple indices like data1, data1_1, data2, ...
• dataview "data1*" (wildcard) to get all the data from data1 & data1_1

We have the following scenario:
Our application is generating new and changed data (More fields & changed fields) so we will have to create a new index with the new mapping to save all that data. We still want to view the old data so our idea is to reindex all the data from data1, data1_1, data2, ... into a single "newdata_1" index. Thats fine and works.
Our question is: How can we switch over to this new index in kibana without downtime. We want to switch over from our multiple indices to this single one which we will access with "newdata_*".

Is this possible and if so how? Our ideas are to create an alias of newdata_1 and call it data1_new so that it will fall into the wildcard of the original dataview but we are unsure if this works. Is there a better solution?

tl;dr: How to switch from one dataview to another with the same data without downtime and without breaking existing visualizations.

Thanks for any ideas

[defalt]

Alias seems to work and will be "caught" by the original dataview. That would be a solution.
But I saw another option where I would just change the index pattern to the new index. I did this but the dashboards seem to be unbothered by this change and still refere to the data from the old indices. Can I force a refresh on their side?

As you can see originalView is still referencing the other indices even though the dataview was changed.

[bhavyarm]

So this is a bit tricky. Your visualizations and saved searches will still be referring to old dataview.

The way to correct it would be to export your saved objects, change the dataview id in the ndjsonto the alias id and then import them again. Here is the screen where you would do it.

We don't recommend this because potential for breakage is high. So if you do it - I suggest to create a new space and experiment and once you got it right - you can copy the corrected saved objects to original space or export and import again.

Thanks,
Bhavya

[defalt]

Thanks for the reply. And whats your opinion on just creating a new index and creating an alias that will be referenced by the dataview? Lets say our dataview is data1* and we create a new index new-index and this new index gets the alias data1_newindex-alias. We tried it and it seems to work (The dataview accepts this index and shows the data, now we delete the old indexes and just have the single new one) but is this a viable solution or is there something we are missing?

[bhavyarm]

@defalt

And whats your opinion on just creating a new index and creating an alias that will be referenced by the dataview? Lets say our dataview is data1* and we create a new index new-index and this new index gets the alias data1_newindex-alias. We tried it and it seems to work (The dataview accepts this index and shows the data, now we delete the old indexes and just have the single new one) but is this a viable solution or is there something we are missing?

Apologies I missed that. Yes this is a viable solution.