Merge 2 indexes in one with same id

Elastic Stack Elasticsearch
1

Hello,

I try to merge 2 indexes into one.
The first index (index_1) looks like this:

{
        "_index" : "index_1",
        "_id" : "1",
        "_source" : {
            "a":"1"
        }
      },
{
        "_index" : "index_1",
        "_id" : "2",
        "_source" : {
            "a":"2"
        }
}

The second index (index_2) looks like this:

  {
        "_index" : "index_2",
        "_id" : "1",
        "_source" : {
            "a":"3"
    }
}

Now I want to merge this two indexes into a new index (index_1_2):

{
        "_index" : "index_1_2",
        "_id" : "1",
        "_source" : {
            "a":"3"
        }
      },
{
        "_index" : "index_1_2",
        "_id" : "2",
        "_source" : {
            "a":"2"
        }
}

In older versions (< 7.0.0) for me the reindex was working:

POST _reindex
{
  "source": {
    "index": ["index_1", "index_2"]
},
  "dest": {
    "index": "index_1_2"
  }
}

But this is not working anymore. In the documentation (https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html)
is for this reindexing the following note:
"The Reindex API makes no effort to handle ID collisions so the last document written will "win" but the order isn’t usually predictable so it is not a good idea to rely on this behavior. Instead, make sure that IDs are unique using a script."

How can I solve this problem?

Thank you!
Oliver

2

Is the assumption that the document with the higher value for a is the one that you want?

3

Hi Mark,

yes, I want in index_1_2 the value a="3" from index_2.
index_2 is a subset of index_1. In the index_2 are only documents which have changes from index_1.

Best regards
Oliver

4

Hello,

does anyone have a solution for that issue?

Best regards
Oliver

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.