Combining indices with same name but different content

Elastic Stack Elasticsearch
1

Good afternoon,

I would like to ask a question regarding a situation that we will face soon in our production environment.

We are moving our Elastic Stack from one server to another and, for some reasons we need to have both of them running simultaneously for about a week.

We have indexes produced on a daily basis and therefore, during each day of simultaneous running, we will have an indexes with the same name (but different documents, no duplicates between them) produced in both servers.
For example we will have a logstash-2017.11.16 created in server1 and in server2.

At some point server1 will be shut down and I will take a snapshot of the indexes to be restored in server2.

My question is: how can I deal with those indexes having the same name?
Shall I treat them separately and merge them apart or the snapshot will correctly append the documents of server1 in the index of server2?

Thanks and regards,
Alessandro

2

Hi,

It's better to treat them separately since merging the data might create conflicts (duplicated data, field names with different values but the same name, etc...).

You can always use aliases which are nice since you can refer to multiple indices or change the backing index with minimal friction.
Assume you have index A on host X and Y. You want to move X to Y. You move index A to Y under X_A, rename existing X to X_B and make an alias X pointing to X_A and X_B.
All apps will still use X which right now points to both A and B. If any app wants to access the old index, you can just bypass the alias.

Hope this helps,

3

Hi Costin,

first of all thanks for your quick reply. I have tested your solution, which works fine.

At the end I have realized that there is an even simpler solution that can be used for our case. Since we are using indices mostly filtering with wild card matching in our searches, dashboards and visualizations (for example logstash-*), the easiest solution would be to take a snapshot from server1 (or host X, using your terminology) and then restore to server2 (or host Y) using the "rename_replacement" option, for appending "_migrated" to the name of each index.

In this way in server2, there will be two kind of indexes: logstash-YYYY.MM.DD and logstash-YYYY.MM.DD_migrated. Therefore filtering on logstash-*, will display the content of both indexes for a generic date YYYY.MM.DD.

Alessandro

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.