How to change _id as GUID during reindex

Elastic Stack Elasticsearch
1

I have two different elastic search nodes.I need to reindex 10 indices from one node to 1 index in another node.But there is not unique id's in the 10 indices, so i need to generate _id as GUID while reindexing

2

Hi Team,
Is the above is possible using elastic search ?

3

Hi,

As I understand your problem you have documents in different index with the same id, and you want to save them in a unique index without loosing your document?

Here 2 documents with similar id in 2 different index

POST twitter/doc/1
{
  "name": "abc"
}

POST twitter2/doc/1
{
  "name": "def"
}

Run _reindex with removing _id from the source so elastic will set a new id
EDIT: you can reindex with one call by settings the list of your source index.

From reindex doc page: Reindex API | Elasticsearch Guide [8.11] | Elastic

index in source can be a list, allowing you to copy from lots of sources in one request.

POST _reindex
{
  "source": {
    "index": ["twitter", "twitter2"]
  },
  "dest": {
    "index": "new_twitter"
  },
  "script": {
    "source": "ctx._id=null",
    "lang": "painless"
  }
}

Running a get on the new index

GET new_twitter/_search

Will return:

  "hits" : {
    "total" : 2,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "new_twitter",
        "_type" : "doc",
        "_id" : "fg6M_WoBShLPuKD8afVc",
        "_score" : 1.0,
        "_source" : {
          "name" : "abc"
        }
      },
      {
        "_index" : "new_twitter",
        "_type" : "doc",
        "_id" : "XQ6M_WoBShLPuKD8p_ka",
        "_score" : 1.0,
        "_source" : {
          "name" : "def"
        }
      }
    ]
  }

According to the documentation:

Think of the possibilities! Just be careful; you are able to change:

  • _id
  • _index
  • _version
  • _routing

Setting _version to null or clearing it from the ctx map is just like not sending the version in an indexing request;

So set ctx._id=null remove the value and on insert Elastic will set a new value.

2 Likes
4

@gabriel_tessier Thank you so much :slightly_smiling_face:

5

Just for information i edited my previous post, replaced the 2 _reindex call with one.

1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.