# Merging search results across multiple related indexes **URL:** \ **Category:** Elasticsearch\ **Created:** [August 20, 2021, 4:33pm UTC](https://discuss.elastic.co/t/merging-search-results-across-multiple-related-indexes/282100 "2021-08-20T16:33:04Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![someguy](https://avatars.discourse-cdn.com/v4/letter/s/ac8455/32.png) [@someguy](https://discuss.elastic.co/u/someguy)\ **Post date:** [August 20, 2021, 4:33pm UTC](https://discuss.elastic.co/t/merging-search-results-across-multiple-related-indexes/282100/1 "2021-08-20T16:33:04Z")
Working on a project where we have a core document (one index) and each document can have multiple related files that we want searchable (a couple other indexes for different types of files). We want to implement a search that spans all indexes, but ideally we only receive results from the the core document index, but the core document index scores reflect matches on the associated file content as well. We have chosen to separate the files into additional indexes due to the variability of the number of related files, and for being able to add/update indexes more simply. Any suggestions? ---
**Author:** ![Sean\_Story](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sean_story/32/69987_2.png) [@Sean\_Story](https://discuss.elastic.co/u/Sean_Story)\ **Post date:** [August 20, 2021, 7:14pm UTC](https://discuss.elastic.co/t/merging-search-results-across-multiple-related-indexes/282100/2 "2021-08-20T19:14:54Z")
Hi @someguy - can you clarify, are you using [Elastic App Search](https://www.elastic.co/app-search/) (in which case you're talking about documents in multiple _engines_) or are you using [Elasticsearch](https://www.elastic.co/elasticsearch/) where you're writing documents directly into indexes? If you're using App Search, have you already looked at [Meta Engines](https://www.elastic.co/guide/en/app-search/current/meta-engines-guide.html)? Or have you considered (App Search or Elasticsearch) having one large document that represents both you core document and has fields to represent the supporting file contents as well, so that you can search over all the related contents in a single index/engine? ---
**Author:** ![someguy](https://avatars.discourse-cdn.com/v4/letter/s/ac8455/32.png) [@someguy](https://discuss.elastic.co/u/someguy)\ **Post date:** [August 20, 2021, 8:36pm UTC](https://discuss.elastic.co/t/merging-search-results-across-multiple-related-indexes/282100/3 "2021-08-20T20:36:43Z")
My bad - we're using Elasticsearch, not App Search. May need to move this to a different product/category? We originally were going with the single large document. That absolutely makes the most sense - search wise. However, due to the significant numbers of documents and their supporting files, we decided to separate them to reduce the burden/load of having to re-index each time one bit of data changes. I've read a little about parent/child relationships and nested documents. It didn't seem like those were options that would help significantly in this particular situation. Our approach was going to have to be to do a search across all indexes, and then do some post-search processing to merge file results to their parent document. We can do that, but its not optimal! ---
**Author:** ![Sean\_Story](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sean_story/32/69987_2.png) [@Sean\_Story](https://discuss.elastic.co/u/Sean_Story)\ **Post date:** [August 23, 2021, 3:40pm UTC](https://discuss.elastic.co/t/merging-search-results-across-multiple-related-indexes/282100/4 "2021-08-23T15:40:54Z")
> [@someguy](#): > > My bad - we're using Elasticsearch, not App Search. May need to move this to a different product/category? Happens all the time, no worries. I've re-categorized it. > [@someguy](#): > > We originally were going with the single large document. That absolutely makes the most sense - search wise. However, due to the significant numbers of documents and their supporting files, we decided to separate them to reduce the burden/load of having to re-index each time one bit of data changes. Did you look at [upserts](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update.html#_update_part_of_a_document) at all? You can provide just the fields that are changing, and not have to re-index all the related content. ---
**Author:** ![someguy](https://avatars.discourse-cdn.com/v4/letter/s/ac8455/32.png) [@someguy](https://discuss.elastic.co/u/someguy)\ **Post date:** [August 24, 2021, 3:28pm UTC](https://discuss.elastic.co/t/merging-search-results-across-multiple-related-indexes/282100/5 "2021-08-24T15:28:24Z")
We have looked at upserts, and, maybe I'm misunderstanding, but it appeared in the documentation that you can't use the ingest pipeline for file indexing. As described - we have a parent document with some data that will change fairly regularly. Each parent document will have 1+ files that we want indexed. The combination of the parent and each related child file document constitutes a single search item. We have to be able to search all of these as a single unit, but we want to be able to index the files separately, and do not want to have to re-index the files when the parent document is modified. Given those requirements, what would the optimal index strategy be? Would nested indexes work, or is there a better approach? Thanks for your assistance! ---
**Author:** ![Sean\_Story](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sean_story/32/69987_2.png) [@Sean\_Story](https://discuss.elastic.co/u/Sean_Story)\ **Post date:** [August 24, 2021, 4:15pm UTC](https://discuss.elastic.co/t/merging-search-results-across-multiple-related-indexes/282100/6 "2021-08-24T16:15:08Z")
Well, we've reached the end of my wisdom in Elasticsearch (I'm an Enterprise Search team member), so I'm going to loop in @dadoonet (who in turn may loop in someone else on the Elasticsearch team) to get you some nuanced guidance. Good luck! ---
**Author:** ![someguy](https://avatars.discourse-cdn.com/v4/letter/s/ac8455/32.png) [@someguy](https://discuss.elastic.co/u/someguy)\ **Post date:** [August 24, 2021, 5:15pm UTC](https://discuss.elastic.co/t/merging-search-results-across-multiple-related-indexes/282100/7 "2021-08-24T17:15:54Z")
Sean - thank you for your assistance. It looks like we are probably going to move to a single index, and look for ways to optimize the indexing of the files into the single document. That will give us the search experience we need. It just changes the focus of our efforts from search to ingest/index time. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [September 3, 2021, 3:00pm UTC](https://discuss.elastic.co/t/merging-search-results-across-multiple-related-indexes/282100/8 "2021-09-03T15:00:15Z")
Thanks @Sean_Story for the ping. @someguy I think it could be a good use case for parent / child feature. See the [join datatype](https://www.elastic.co/guide/en/elasticsearch/reference/current/parent-join.html). In that case you could always index and update parent document without needing to reindex the child docs. Would that work for you? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [September 3, 2021, 3:01pm UTC](https://discuss.elastic.co/t/merging-search-results-across-multiple-related-indexes/282100/9 "2021-09-03T15:01:45Z")
I agree that sometime it is better to reindex everything if it happens occasionally vs trying to use parent/child which comes with some complexity and hidden costs (like memory usage, single shard location...) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 1, 2021, 3:02pm UTC](https://discuss.elastic.co/t/merging-search-results-across-multiple-related-indexes/282100/10 "2021-10-01T15:02:02Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.