# Aggregate Score for Hybrid Search

**URL:** <https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205>\
**Category:** Elasticsearch\
**Tags:** vector-search\
**Created:** [February 10, 2023, 12:02am UTC](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205 "2023-02-10T00:02:53Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kok\_Gin\_Xian](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kok_gin_xian/32/117084_2.png) [@Kok\_Gin\_Xian](https://discuss.elastic.co/u/Kok_Gin_Xian)\
**Post date:** [February 10, 2023, 12:02am UTC](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205/1 "2023-02-10T00:02:53Z")

</div>

Hi, I'm new to Elasticsearch and am trying out the new hybrid search by specifying the "knn" and "query" parameters in my search.

I set k=100 in knn, size=k=100 in the search request.

For pure vector search (omitting the "query" parameter), I got 100 hits ranked according to the cosine similarity score. This is consistent with my expectations.

However, with hybrid search (adding the "query" parameter with a match-all constant score filter with a boost of 1.0 for example, only the first 25 hits \_scores are (1.0 + cosine similarity score), the remaining 75 hits have \_scores of 1.0. Changing the "k" parameter in knn does not change this behavior.

```auto
query={
            "constant_score": {
                "filter": {
                    "match_all": {}
                },
                "boost": 1.0
            }
        }

knn={
            "field": "image_vector",
            "query_vector": query_vector,
            "k": k,
            "num_candidates": 100,
        }

```

What am I doing wrong? I'm using Elasticsearch 8.6.1. Please help. I'm using the Python Elasticsearch client v8.6.1 for search.

---

<div class="post-metadata">

**Author:** ![Kok\_Gin\_Xian](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kok_gin_xian/32/117084_2.png) [@Kok\_Gin\_Xian](https://discuss.elastic.co/u/Kok_Gin_Xian)\
**Post date:** [February 10, 2023, 12:29am UTC](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205/2 "2023-02-10T00:29:28Z")

</div>

Also, the number of KNN results in the hybrid search hits varies depending on what is searched (the query vector used in KNN). This number is not always the set k value. This condition can be easily seen when there is a "large" gap in the hit \_scores when hybrid search is used.

---

<div class="post-metadata">

**Author:** ![BenTrent](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bentrent/32/33915_2.png) [@BenTrent](https://discuss.elastic.co/u/BenTrent)\
**Post date:** [February 10, 2023, 6:18pm UTC](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205/3 "2023-02-10T18:18:32Z")

</div>

Hey @Kok_Gin_Xian ,

These are indeed interesting results. I will try to replicate them myself. Is there a test dataset that replicates your results?

Also, when you don't have the `constant_score` filter, and you are able to see all the "k" values, are any of those scores negative?

Thanks!

---

<div class="post-metadata">

**Author:** ![Kok\_Gin\_Xian](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kok_gin_xian/32/117084_2.png) [@Kok\_Gin\_Xian](https://discuss.elastic.co/u/Kok_Gin_Xian)\
**Post date:** [February 10, 2023, 6:41pm UTC](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205/4 "2023-02-10T18:41:37Z")

</div>

Omitting "query" for constant scoring, I am able to get all the k nearest neighbors . I used cosine similarity and the score for all the k neighbors are in the range of 0.60 - 0.67. I didn't see any negative values. My dataset size is sizable 5m so I think that is also why my cosine scores are quite high.

However, adding the "query" for hybrid search, I see the top few hits ~1.6, then a sudden drop to the constant score 1.0. And the number of hits with 1.x are different for different query vector but repeatable for the same query vector.

---

<div class="post-metadata">

**Author:** ![Kok\_Gin\_Xian](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kok_gin_xian/32/117084_2.png) [@Kok\_Gin\_Xian](https://discuss.elastic.co/u/Kok_Gin_Xian)\
**Post date:** [February 10, 2023, 6:42pm UTC](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205/5 "2023-02-10T18:42:55Z")

</div>

I also tried different versions of Elasticsearch 8.6.0 and 8.5.3 and see no improvement.

---

<div class="post-metadata">

**Author:** ![BenTrent](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bentrent/32/33915_2.png) [@BenTrent](https://discuss.elastic.co/u/BenTrent)\
**Post date:** [February 10, 2023, 8:04pm UTC](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205/6 "2023-02-10T20:04:29Z")

</div>

@Kok_Gin_Xian I am trying to replicate the issue myself and am not having any luck.

What are your number of shards in the index?

Does the index have any deleted documents?

`GET _cat/indices?v` should give you shard count and number of documents & deleted docs.

Does EVERY document have a vector field? Or are some documents missing the vector field?

To figure this out, you can do an [Exists query | Elasticsearch Guide [8.6] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html)

---

<div class="post-metadata">

**Author:** ![Kok\_Gin\_Xian](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kok_gin_xian/32/117084_2.png) [@Kok\_Gin\_Xian](https://discuss.elastic.co/u/Kok_Gin_Xian)\
**Post date:** [February 11, 2023, 2:47am UTC](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205/7 "2023-02-11T02:47:03Z")

</div>

> [@BenTrent](#):
>
> GET \_cat/indices?v

Hi, this is the output from the request.

'health status index uuid pri rep docs.count docs.deleted store.size pri.store.size\nyellow open image-index AWtVrHStQ3GDJLNoUToAXQ 1 1 3056822 0 29.8gb 29.8gb\n'

---

<div class="post-metadata">

**Author:** ![Kok\_Gin\_Xian](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kok_gin_xian/32/117084_2.png) [@Kok\_Gin\_Xian](https://discuss.elastic.co/u/Kok_Gin_Xian)\
**Post date:** [February 11, 2023, 3:29am UTC](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205/9 "2023-02-11T03:29:52Z")

</div>

All data samples should be complete, but let me know if there is a way to check in Elasticsearch.

---

<div class="post-metadata">

**Author:** ![Kok\_Gin\_Xian](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kok_gin_xian/32/117084_2.png) [@Kok\_Gin\_Xian](https://discuss.elastic.co/u/Kok_Gin_Xian)\
**Post date:** [February 13, 2023, 3:51am UTC](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205/10 "2023-02-13T03:51:07Z")

</div>

I've retried the inserting of data and can confirm all the fields for the samples are complete, including the vector data.

I also casted the data type of the vector data to float32 and it didn't help with the problem.

---

<div class="post-metadata">

**Author:** ![Kok\_Gin\_Xian](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kok_gin_xian/32/117084_2.png) [@Kok\_Gin\_Xian](https://discuss.elastic.co/u/Kok_Gin_Xian)\
**Post date:** [February 14, 2023, 8:13am UTC](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205/11 "2023-02-14T08:13:29Z")

</div>

@BenTrent , wondering if you managed to try my simple image search web app? I don't know how to proceed from here.

---

<div class="post-metadata">

**Author:** ![BenTrent](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bentrent/32/33915_2.png) [@BenTrent](https://discuss.elastic.co/u/BenTrent)\
**Post date:** [February 14, 2023, 12:51pm UTC](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205/12 "2023-02-14T12:51:12Z")

</div>

@Kok_Gin_Xian that doesn't really help me debug. I am still looking into it. I still cannot replicate.

The easiest way to debug is to attempt to create a minimal working replication of the issue.

@Kok_Gin_Xian does the same problem occur if you use the Kibana dev console for your search?

Additionally, on one of the failing documents (the ones that show up as just 1.0 in the middle of the list), can you call the explain API with your hybrid search? [Explain API | Elasticsearch Guide [8.6] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-explain.html)

---

<div class="post-metadata">

**Author:** ![BenTrent](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bentrent/32/33915_2.png) [@BenTrent](https://discuss.elastic.co/u/BenTrent)\
**Post date:** [February 14, 2023, 8:54pm UTC](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205/13 "2023-02-14T20:54:28Z")

</div>

> Additionally, on one of the failing documents (the ones that show up as just 1.0 in the middle of the list), can you call the explain API with your hybrid search? [Explain API | Elasticsearch Guide [8.6] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-explain.html)

This is incorrect. Could you make your failing call but with `"explain": True` in the search body as well? It will output a ton, and may take longer but it will indicate the query clauses that score on that document hit.

---

<div class="post-metadata">

**Author:** ![Kok\_Gin\_Xian](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kok_gin_xian/32/117084_2.png) [@Kok\_Gin\_Xian](https://discuss.elastic.co/u/Kok_Gin_Xian)\
**Post date:** [February 15, 2023, 3:20am UTC](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205/14 "2023-02-15T03:20:58Z")

</div>

Ok, let me prepare you a minimal application for reproducing the problem.

---

<div class="post-metadata">

**Author:** ![Kok\_Gin\_Xian](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kok_gin_xian/32/117084_2.png) [@Kok\_Gin\_Xian](https://discuss.elastic.co/u/Kok_Gin_Xian)\
**Post date:** [February 15, 2023, 3:33am UTC](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205/15 "2023-02-15T03:33:48Z")

</div>

This is the csv containing the \_score and \_explanation for the hits.

```auto
_score,_explanation
1.6622047,"{'value': 1.6622047, 'description': 'sum of:', 'details': [{'value': 0.66220474, 'description': 'within top k documents', 'details': []}, {'value': 1.0, 'description': 'ConstantScore(*:*)', 'details': []}]}"
1.6613073,"{'value': 1.6613073, 'description': 'sum of:', 'details': [{'value': 0.6613073, 'description': 'within top k documents', 'details': []}, {'value': 1.0, 'description': 'ConstantScore(*:*)', 'details': []}]}"
1.6607261,"{'value': 1.6607261, 'description': 'sum of:', 'details': [{'value': 0.660726, 'description': 'within top k documents', 'details': []}, {'value': 1.0, 'description': 'ConstantScore(*:*)', 'details': []}]}"
1.6599699,"{'value': 1.6599699, 'description': 'sum of:', 'details': [{'value': 0.6599699, 'description': 'within top k documents', 'details': []}, {'value': 1.0, 'description': 'ConstantScore(*:*)', 'details': []}]}"
1.6598208,"{'value': 1.6598208, 'description': 'sum of:', 'details': [{'value': 0.65982085, 'description': 'within top k documents', 'details': []}, {'value': 1.0, 'description': 'ConstantScore(*:*)', 'details': []}]}"
1.0,"{'value': 1.0, 'description': 'sum of:', 'details': [{'value': 1.0, 'description': 'ConstantScore(*:*)', 'details': []}]}"
1.0,"{'value': 1.0, 'description': 'sum of:', 'details': [{'value': 1.0, 'description': 'ConstantScore(*:*)', 'details': []}]}"
1.0,"{'value': 1.0, 'description': 'sum of:', 'details': [{'value': 1.0, 'description': 'ConstantScore(*:*)', 'details': []}]}"
1.0,"{'value': 1.0, 'description': 'sum of:', 'details': [{'value': 1.0, 'description': 'ConstantScore(*:*)', 'details': []}]}"
1.0,"{'value': 1.0, 'description': 'sum of:', 'details': [{'value': 1.0, 'description': 'ConstantScore(*:*)', 'details': []}]}"

```

This is the src for the search in Python:

```auto
text = "lion"

text_embed = encode_text(text)
query_vector = text_embed.tolist()

k = 10

resp = es.search(
    index="image-index",
    size=k,
    request_timeout=30,
    query={"constant_score": {"filter": {"match_all": {}}, "boost": 1.0}},
    knn={
        "field": "image_vector",
        "query_vector": query_vector,
        "k": k,
        "num_candidates": 100,
    },
    _source=["image_url", "image_desc", "filetype"],
    explain=True
)

```

---

<div class="post-metadata">

**Author:** ![Kok\_Gin\_Xian](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kok_gin_xian/32/117084_2.png) [@Kok\_Gin\_Xian](https://discuss.elastic.co/u/Kok_Gin_Xian)\
**Post date:** [February 15, 2023, 9:17am UTC](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205/16 "2023-02-15T09:17:06Z")

</div>

Please see this GitHub repo for a minimal reproducible application.

> **[GitHub - xiankgx/es-debug](https://github.com/xiankgx/es-debug)**
>
> Contribute to xiankgx/es-debug development by creating an account on GitHub.

---

<div class="post-metadata">

**Author:** ![Kok\_Gin\_Xian](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kok_gin_xian/32/117084_2.png) [@Kok\_Gin\_Xian](https://discuss.elastic.co/u/Kok_Gin_Xian)\
**Post date:** [February 15, 2023, 9:40am UTC](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205/17 "2023-02-15T09:40:50Z")

</div>

The explanation I get for the hits blew my mind and I don't understand one bit what it is doing.

> <https://github.com/xiankgx/es-debug/blob/main/debug_hybrid_search.ipynb>

---

<div class="post-metadata">

**Author:** ![BenTrent](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bentrent/32/33915_2.png) [@BenTrent](https://discuss.elastic.co/u/BenTrent)\
**Post date:** [February 15, 2023, 12:49pm UTC](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205/18 "2023-02-15T12:49:08Z")

</div>

Thank you so much @Kok_Gin_Xian for all the info! I will dig in and report back!

---

<div class="post-metadata">

**Author:** ![BenTrent](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bentrent/32/33915_2.png) [@BenTrent](https://discuss.elastic.co/u/BenTrent)\
**Post date:** [February 15, 2023, 3:17pm UTC](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205/19 "2023-02-15T15:17:01Z")

</div>

@Kok_Gin_Xian I was able to replicate! Thanks!

This behavior is indeed surprising. I will see if I can find the cause.

Here is the bug: [Vector search hybrid score surprising behavior · Issue #93830 · elastic/elasticsearch · GitHub](https://github.com/elastic/elasticsearch/issues/93830)

I will be working on figuring out the cause and seeing about a fix.

Thank you so much for the replication data and steps! It makes this all much simpler.

---

<div class="post-metadata">

**Author:** ![Kok\_Gin\_Xian](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kok_gin_xian/32/117084_2.png) [@Kok\_Gin\_Xian](https://discuss.elastic.co/u/Kok_Gin_Xian)\
**Post date:** [February 16, 2023, 12:15am UTC](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205/20 "2023-02-16T00:15:38Z")

</div>

I'm glad you are able to replicate the problem. Thank you so much for the help in replicating the problem and working on the bug!

---

<div class="post-metadata">

**Author:** ![BenTrent](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bentrent/32/33915_2.png) [@BenTrent](https://discuss.elastic.co/u/BenTrent)\
**Post date:** [February 16, 2023, 3:45pm UTC](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205/21 "2023-02-16T15:45:21Z")

</div>

Hey @Kok_Gin_Xian

I experimented and a current work around could be force-merging the index to a single segment.

> **[Force merge API | Elasticsearch Guide \[8.6\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-forcemerge.html)**

So

```auto
POST <index>/_forcemerge?max_num_segments=1

```

I am still digging into what the exact cause is over multiple segments in the same shard.

Also, I did find a bug with `explain` that i will be fixing as well.

Thank you for helping make Elasticsearch better!!!

[Next page](https://discuss.elastic.co/t/aggregate-score-for-hybrid-search/325205.md?page=2)