# Multiple Indices - Index Documents with same ID - Treat as single document when aggregating

**URL:** https://discuss.elastic.co/t/multiple-indices-index-documents-with-same-id-treat-as-single-document-when-aggregating/196754
**Category:** Elasticsearch
**Created:** [August 26, 2019, 11:39am UTC](https://discuss.elastic.co/t/multiple-indices-index-documents-with-same-id-treat-as-single-document-when-aggregating/196754 "2019-08-26T11:39:19Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![gnukev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gnukev/32/53042_2.png) [@gnukev](https://discuss.elastic.co/u/gnukev)
#### Post date: [August 26, 2019, 11:39am UTC](https://discuss.elastic.co/t/multiple-indices-index-documents-with-same-id-treat-as-single-document-when-aggregating/196754/1 "2019-08-26T11:39:19Z")

</div>

Would the following be a possibility?

# Index A - Document A

{  
\_id: 123456789,  
name: XXXX,  
address: XXXX  
}

# Index B - Document A

{  
\_id: 123456789,  
biography: XXXXXyyyXXXXyyyyXXXXyyyyXXXX  
}

And then I do a terms aggregation against both indices on "name" where "biography" has a certain text.

Basically, is it feasible in ES to separate out large text fields into separate documents in multiple indices but treat them as the same document if they have the same ID when doing searches or running aggregations?

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [August 26, 2019, 11:56am UTC](https://discuss.elastic.co/t/multiple-indices-index-documents-with-same-id-treat-as-single-document-when-aggregating/196754/2 "2019-08-26T11:56:27Z")

</div>

No, that is not possible. What is the problem you are trying to solve?

---

<div class="post-metadata">

### Author: ![gnukev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gnukev/32/53042_2.png) [@gnukev](https://discuss.elastic.co/u/gnukev)
#### Post date: [August 26, 2019, 12:56pm UTC](https://discuss.elastic.co/t/multiple-indices-index-documents-with-same-id-treat-as-single-document-when-aggregating/196754/3 "2019-08-26T12:56:47Z")

</div>

Thanks @Christian_Dahlqvist for gettting back.

I did try and it was not possible. I was thinking if this could be a potential future consideration.

The problem I was looking to solve with this was the following scenario:

1. I have a document mapping where many fields are basic fields with less text content.

2. Only a couple of fields are large text fields and infrequently used for full text searches against that field and running aggregations on other fields with the full text search filter.

3. If I could store them separately and use them only when required, the overall document size would be smaller and it would save on maintenance as the large field is always the same for that particular document. It would help with updates to the data as well as I need to only focus on a smaller document.

Not sure if this is feasible though or the negative effects it will have elsewhere.

Thanks!

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [August 26, 2019, 1:03pm UTC](https://discuss.elastic.co/t/multiple-indices-index-documents-with-same-id-treat-as-single-document-when-aggregating/196754/4 "2019-08-26T13:03:54Z")

</div>

If you have documents with a large component that does not change frequently and some fields that are updated more regularly I have seen a similar arrangement to what you described but using [parent-child relationship](https://www.elastic.co/guide/en/elasticsearch/reference/7.3/parent-join.html) within a single index. This allows you to update the documents independently and allows you to aggregate over them, although possibly with a different syntax.

---

<div class="post-metadata">

### Author: ![gnukev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gnukev/32/53042_2.png) [@gnukev](https://discuss.elastic.co/u/gnukev)
#### Post date: [August 26, 2019, 1:17pm UTC](https://discuss.elastic.co/t/multiple-indices-index-documents-with-same-id-treat-as-single-document-when-aggregating/196754/5 "2019-08-26T13:17:02Z")

</div>

Thanks @Christian_Dahlqvist.

I did look at parent-child relationships within a single index but it mentioned "The only case where the join field makes sense is if your data contains a one-to-many relationship where one entity significantly outnumbers the other entity" and also mentioned that it has significant performance impacts to your search query.

The document I have with the large text field is a one-to-one unique text for that document. So I was thinking if it would be possible to consider the above scenario for the future or if there are any outright considerable limitations to it. I wanted to make minimal changes to my queries as well if that was feasible.

Thanks!

---

<div class="post-metadata">

### Author: ![gnukev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gnukev/32/53042_2.png) [@gnukev](https://discuss.elastic.co/u/gnukev)
#### Post date: [August 27, 2019, 9:33am UTC](https://discuss.elastic.co/t/multiple-indices-index-documents-with-same-id-treat-as-single-document-when-aggregating/196754/6 "2019-08-27T09:33:16Z")

</div>

Any other thoughts @Christian_Dahlqvist? Could this be considered as a use-case?

---

<div class="post-metadata">

### Author: ![gnukev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gnukev/32/53042_2.png) [@gnukev](https://discuss.elastic.co/u/gnukev)
#### Post date: [September 17, 2019, 9:56am UTC](https://discuss.elastic.co/t/multiple-indices-index-documents-with-same-id-treat-as-single-document-when-aggregating/196754/7 "2019-09-17T09:56:16Z")

</div>

@Christian_Dahlqvist or anybody else? I'd love to hear back on the feasibility of the request.

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [September 17, 2019, 11:48am UTC](https://discuss.elastic.co/t/multiple-indices-index-documents-with-same-id-treat-as-single-document-when-aggregating/196754/8 "2019-09-17T11:48:59Z")

</div>

I would say it is feasible. Using parent-child to avoid repeatedly reindexing a large document could work even though it is not the typical use case. Whether it is worth it or not I think you need to test to determine.

---

<div class="post-metadata">

### Author: ![gnukev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gnukev/32/53042_2.png) [@gnukev](https://discuss.elastic.co/u/gnukev)
#### Post date: [September 18, 2019, 1:08pm UTC](https://discuss.elastic.co/t/multiple-indices-index-documents-with-same-id-treat-as-single-document-when-aggregating/196754/9 "2019-09-18T13:08:46Z")

</div>

Thanks, @Christian_Dahlqvist.

Ok, but no way to aggregate from two indices using the same document ID? I mean I am not asking in the current version itself but potentially as a future consideration for the benefits it would have.

Or are there more technical challenges and cons than the potential benefits or is it completely not possible? Please let me know your thoughts.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 16, 2019, 1:09pm UTC](https://discuss.elastic.co/t/multiple-indices-index-documents-with-same-id-treat-as-single-document-when-aggregating/196754/10 "2019-10-16T13:09:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
