# Complex similary search with "control group"

**URL:** <https://discuss.elastic.co/t/complex-similary-search-with-control-group/329610>\
**Category:** Elastic Search\
**Tags:** elastic-app-search\
**Created:** [April 8, 2023, 10:47am UTC](https://discuss.elastic.co/t/complex-similary-search-with-control-group/329610 "2023-04-08T10:47:28Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![MikeT1234](https://avatars.discourse-cdn.com/v4/letter/m/8797f3/32.png) [@MikeT1234](https://discuss.elastic.co/u/MikeT1234)\
**Post date:** [April 8, 2023, 10:47am UTC](https://discuss.elastic.co/t/complex-similary-search-with-control-group/329610/1 "2023-04-08T10:47:28Z")

</div>

I'm trying to understand how should I implement following search feature.

User can add companies to a control group (for example 100 companies) and search similar companies (max 300) based on  
criteria groups like location (region, municipality), industry, revenue and some other fields.  
Location documents contain official code (string) + name.  
Financial fields like revenue are numeric.

Result should contain scores from different criteria groups so that data can be used in the application' s UI. User should how well location/industry/revenue is matched to the control group and total score.

Could somebody point me to the right direction how this could be done?

---

<div class="post-metadata">

**Author:** ![MikeT1234](https://avatars.discourse-cdn.com/v4/letter/m/8797f3/32.png) [@MikeT1234](https://discuss.elastic.co/u/MikeT1234)\
**Post date:** [April 11, 2023, 9:42pm UTC](https://discuss.elastic.co/t/complex-similary-search-with-control-group/329610/2 "2023-04-11T21:42:58Z")

</div>

I would appreciate you comments about this 😜

---

<div class="post-metadata">

**Author:** ![MikeT1234](https://avatars.discourse-cdn.com/v4/letter/m/8797f3/32.png) [@MikeT1234](https://discuss.elastic.co/u/MikeT1234)\
**Post date:** [April 14, 2023, 11:33am UTC](https://discuss.elastic.co/t/complex-similary-search-with-control-group/329610/3 "2023-04-14T11:33:49Z")

</div>

Any ideas?

---

<div class="post-metadata">

**Author:** ![Kathleen\_DeRusso](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kathleen_derusso/32/132039_2.png) [@Kathleen\_DeRusso](https://discuss.elastic.co/u/Kathleen_DeRusso)\
**Post date:** [April 18, 2023, 6:59pm UTC](https://discuss.elastic.co/t/complex-similary-search-with-control-group/329610/4 "2023-04-18T18:59:59Z")

</div>

Hi @MikeT1234 - Thanks for your question!

Assuming that the control group and other companies are in the same index (or different indexes with the same alias), you could index flags like `is_control_group` as a field in your document alongside your criteria fields like `location`, `industry`, etc. This way you could easily exclude your control documents from your search results.

From the sounds of it, you may be interested in experimenting with the [More Like This](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-mlt-query.html) query. You can use the More Like This query to specify specific fields (your criteria) that you want to pull similar results for, and even compare them with specific documents in your index.

I'm not quite sure how feasible it would be to get a lot of different scores in a single API response call though - at least not in any way that would be remotely performant. It might be easier to return the search results with their criteria values, and let your UI use rules to highlight the criteria values that matched in a way that makes sense.

Good luck, I'd be interested to know if this strategy works for you!

---

<div class="post-metadata">

**Author:** ![MikeT1234](https://avatars.discourse-cdn.com/v4/letter/m/8797f3/32.png) [@MikeT1234](https://discuss.elastic.co/u/MikeT1234)\
**Post date:** [April 20, 2023, 10:44am UTC](https://discuss.elastic.co/t/complex-similary-search-with-control-group/329610/5 "2023-04-20T10:44:27Z")

</div>

Hi Kathleen. Thanks for your input 🙂

Using` more_like_this` - query was one of my first idea how to solve the problem.  
According to the documentation I got the impression that it works mainly with string -fields and not with numeric field (+range condition) so I rejected this idea.

Another idea was to make a search per category (location, industry, revenue etc) since number of categories is not that high (max 4) -\> it sounded feasible.  
Problem here was that I couldn't figure out how to combine these results / how to get the union (max 300 best results), so I rejected this idea.

So far my best idea is to do one query/search using `minimum_should_match`. It should return best results + do category scoring without Elasticsearch at the back-end side.  
It sounded like you ended up to a similar resolution.

I really appreciated your comment. Elasticsearch's documentation is quite vast and I wasn't sure if missed something important 🙂

---

<div class="post-metadata">

**Author:** ![Kathleen\_DeRusso](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kathleen_derusso/32/132039_2.png) [@Kathleen\_DeRusso](https://discuss.elastic.co/u/Kathleen_DeRusso)\
**Post date:** [April 20, 2023, 9:18pm UTC](https://discuss.elastic.co/t/complex-similary-search-with-control-group/329610/6 "2023-04-20T21:18:33Z")

</div>

minimum\_should\_match should work to return only documents that meet your minimum allowed set of matched fields - and you're definitely on the right track that combining results of different searches is a very hard problem as the scores are very different.

The other thing you could potentially find useful is using [function\_score](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#query-dsl-function-score-query) to boost, say, revenue closest to the average of the control group (but that would probably have to be known at query time).

Good luck!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 18, 2023, 9:18pm UTC](https://discuss.elastic.co/t/complex-similary-search-with-control-group/329610/7 "2023-05-18T21:18:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
