Add any 2 Index to form new index which will combined data and should not override any fields values

Elastic Stack Kibana
1

Hello All,

I want to merge two Index with same structure , which will create 3rd Index without overriding field values.
For example:
index_1:
"businessUnits": "FR,JP,IE"

index_2:
"businessUnits": "FR,GB,DE"

If I merge index_1 and index_2 using reindex API, It should not override values.
Ii should give output as : "businessUnits": "FR,JP,IE,FR,GB,DE"

PUT emea_test/_doc/871123365V

{
  "type": "db-poc-user",
  "skuNum": " 871123365V",
  "businessUnits": "FR,JP,IE",
  "businessUnitsWithNames": [
    "[FR] France",
    "[JP] Japan",
    "[IE] Ireland"
  ],
  "@timestamp": "2020-07-22T04:12:00.376Z"
}

 ==================================

 

PUT global_test/_doc/871123365V
{
  "type": "db-test-user",
  "skuNum": " 871123365V",
  "businessUnits": "FR,GB,DE",
  "businessUnitsWithNames": [
    "[FR] France",
    "[GB] UK",
    "[DE] Germany"
  ],
  "@timestamp": "2020-07-23T04:12:00.376Z"
}

Output should be (concatenate):

"skuNum": " 871123365V",
"businessUnits": "FR,JP,IE,FR,GB,DE"
  "businessUnitsWithNames": [
    "[FR] France",
    "[JP] Japan",
    "[IE] Ireland",
"[FR] France",
    "[GB] UK",
    "[DE] Germany"
  ],

Please help me with this use case

2

@warkolm @stephenb Can you please help me?
Thanks in advance!

3

Hi @Mangesh_Mathe Please try not to @ people that are not in your conversation it is not community best practice.

As far as I can tell, neither reindex nor update_by_query will accomplish what you want I think you are going to need to write some custom code to do what you want.

Your use case requires that both documents _sources are accessed at the same time so they can be "merged" ... neither reindex nor update_by_query have that capability as far as I know.

Perhaps someone else will know how.

But I think you are going to need to write some code that grabs both docs and then merges them to your spec and then writes the new merged doc.

4

how many document do you have in them, 100 - 1000 or millions?
do they have uniq key "skuNum" ?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.