# I am trying to use debezium in a setup where each table is its own index. I want to create a merged index

**URL:** <https://discuss.elastic.co/t/i-am-trying-to-use-debezium-in-a-setup-where-each-table-is-its-own-index-i-want-to-create-a-merged-index/370700>\
**Category:** Elasticsearch\
**Created:** [November 18, 2024, 2:37pm UTC](https://discuss.elastic.co/t/i-am-trying-to-use-debezium-in-a-setup-where-each-table-is-its-own-index-i-want-to-create-a-merged-index/370700 "2024-11-18T14:37:05Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![randomuser24](https://avatars.discourse-cdn.com/v4/letter/r/919ad9/32.png) [@randomuser24](https://discuss.elastic.co/u/randomuser24)\
**Post date:** [November 18, 2024, 2:37pm UTC](https://discuss.elastic.co/t/i-am-trying-to-use-debezium-in-a-setup-where-each-table-is-its-own-index-i-want-to-create-a-merged-index/370700/1 "2024-11-18T14:37:05Z")

</div>

In the debezium setup i'll have an index for each table. Lets say I have the three sources indicies an Owner index, a pets index, treats index that each pet eats. How can I have a merged index where it will include a owner and the pets they own, the treats the pets eat.

I understand we can use the enrichment policies but is there a way to do a merged index. The messages will only hit the three source indexes.

Example

```auto
Owner index

[
  {
    "_index": "owner",
    "_type": "_doc",
    "_id": "1",
    "_score": 1.0,
    "_source": {
      "owner_id": 1,
      "first_nm": "John",
      "last_nm": "Doe"
    }
  },
  {
    "_index": "owner",
    "_type": "_doc",
    "_id": "2",
    "_score": 1.0,
    "_source": {
      "owner_id": 2,
      "first_nm": "Jane",
      "last_nm": "Doe"
    }
  }
]

Pet index
[
  {
    "_index": "pets",
    "_type": "_doc",
    "_id": "1",
    "_score": 1.0,
    "_source": {
      "owner_id": 1,
      "pet_nm": "Bolt",
      "pet_id": 1
    }
  },
  {
    "_index": "pets",
    "_type": "_doc",
    "_id": "2",
    "_score": 1.0,
    "_source": {
      "owner_id": 1,
      "pet_nm": "Sparky",
      "pet_id": 2
    }
  },
  {
    "_index": "pets",
    "_type": "_doc",
    "_id": "3",
    "_score": 1.0,
    "_source": {
      "owner_id": 2,
      "pet_nm": "Yellow",
      "pet_id": 3
    }
  },
  {
    "_index": "pets",
    "_type": "_doc",
    "_id": "4",
    "_score": 1.0,
    "_source": {
      "owner_id": 2,
      "pet_nm": "Winnie",
      "pet_id": 4
    }
  }
]

Treats
[
  {
    "_index": "treats",
    "_type": "_doc",
    "_id": "1",
    "_score": 1.0,
    "_source": {
      "treat_id": 1,
      "food_nm": "decent food",
      "pet_id": 1
    }
  },
  {
    "_index": "treats",
    "_type": "_doc",
    "_id": "2",
    "_score": 1.0,
    "_source": {
      "treat_id": 2,
      "food_nm": "alright food",
      "pet_id": 1
    }
  },
  {
    "_index": "treats",
    "_type": "_doc",
    "_id": "3",
    "_score": 1.0,
    "_source": {
      "treat_id": 3,
      "treat_nm": "best food",
      "pet_id": 2
    }
  },
  {
    "_index": "treats",
    "_type": "_doc",
    "_id": "4",
    "_score": 1.0,
    "_source": {
      "treat_id": 4,
      "food_nm": "ok food",
      "pet_id": 2
    }
  }
]

Merged index needed

[
  {
    "_index": "owner",
    "_type": "_doc",
    "_id": "1",
    "_score": 1.0,
    "_source": {
      "owner_id": 1,
      "first_nm": "John",
      "last_nm": "Doe",
      "pets": [
        {
          "pet_id": 1,
          "pet_nm": "Sparky"
        },
        {
          "pet_id": 2,
          "pet_nm": "Sparky"
        }
      ],
      "treat": [
        {
          "pet_id": 1,
          "treat_id": 1,
          "food_nm": "decent_food"
        },
        {
          "pet_id": 2,
          "treat_id": 2,
          "food_nm": "alright food"
        },
      ]
    }
  } etc..
]

```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [November 18, 2024, 3:14pm UTC](https://discuss.elastic.co/t/i-am-trying-to-use-debezium-in-a-setup-where-each-table-is-its-own-index-i-want-to-create-a-merged-index/370700/2 "2024-11-18T15:14:18Z")

</div>

This seems like a question for the Debezium community as there is no way to merge indices the way you describe (possibly apart from using enrichment policies) within Elasticsearch.

---

<div class="post-metadata">

**Author:** ![randomuser24](https://avatars.discourse-cdn.com/v4/letter/r/919ad9/32.png) [@randomuser24](https://discuss.elastic.co/u/randomuser24)\
**Post date:** [November 18, 2024, 3:19pm UTC](https://discuss.elastic.co/t/i-am-trying-to-use-debezium-in-a-setup-where-each-table-is-its-own-index-i-want-to-create-a-merged-index/370700/3 "2024-11-18T15:19:25Z")

</div>

Will do on messaging the debezium community. Is there a possibility to use the transform api to get something like I mentioned in the topic above. I just made an edit.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [November 18, 2024, 3:34pm UTC](https://discuss.elastic.co/t/i-am-trying-to-use-debezium-in-a-setup-where-each-table-is-its-own-index-i-want-to-create-a-merged-index/370700/4 "2024-11-18T15:34:18Z")

</div>

> [@randomuser24](#):
>
> Is there a possibility to use the transform api to get something like I mentioned in the topic above.

No, you want something like a JOIN, which does not exist in Elasticsearch.

You need to normalize your data before indexing it.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [November 18, 2024, 4:02pm UTC](https://discuss.elastic.co/t/i-am-trying-to-use-debezium-in-a-setup-where-each-table-is-its-own-index-i-want-to-create-a-merged-index/370700/5 "2024-11-18T16:02:51Z")

</div>

It sounds like you want to replicate a join or materialized view of some sorts. This is in my opinion something that should be done before data is ingested into Elasticsearch.
