RRF score in Hybrid Search

Elastic Stack Elasticsearch
1

Hi Team,
I'm working on hybrid search using elastic. Below is the code snippet I'm using-

search_query = {
        "size": k,
        "query": {
            'bool': {
                'should': {
                    'multi_match': {
                        'query': search_term,
                        'fields': ['url', 'text_content'],
                    }
                },
               
            }
        },
       

        "knn": {
            "field": "ml.inference.main_article_vector.predicted_value",
            "query_vector_builder": {
                "text_embedding": {
                    "model_id": ".multilingual-e5-small_linux-x86_64",
                    "model_text": search_term
                }
            },
            "k": k,
            "num_candidates": 10000
        },
        "rank":{"rrf":  {"window_size": k}},
        "fields": [
            "id",
            "url",
            "main_article_text",
            "document_image"
        ],
        "_source": "false"
        
    }

I'm not able to get the final score of the hybrid search. "_score" is None.
Is there a way to get the score along with the "_rank".
Thanks. Looking forward for the resolution.

2

Hi @Shilpi95

I think that with more details like which version of elasticsearch you use and what your query returns as a response would help to understand a little more.

3

@RabBit_BR

Elasticsearch version is Version: 8.15.1
Here's the response which the query returns:

{'_index': 'search-collection',
  '_id': '66e2adc39c1f65f51ffe0d',
  '_score': None,
  '_rank': 1,
  '_ignored': ['main_article_text.keyword'],
  '_source': {},
  'fields': {'id': ['66e2adc39c1f65f51ffe0d'],
   'document_image': ['https://assets.com/media/media.jpg'],
   'url': ['https://sample_url.html'],
   'main_article_text': ['sample text']}},
4

RRF only cares about rank and score is not available in the response. However, better explainability has been added.

5

Thanks @Kathleen_DeRusso for the response. Is there any way I can use custom function for RRF score in the search query?

6

@Kathleen_DeRusso I tried to include the "annotate_source" but getting Bad Request error.

search_query = {
        "size": k,
        "query": {
            'bool': {
                'should': {
                    'multi_match': {
                        'query': search_term,
                        'fields': ['url', 'text_content'],
                    }
                },
               
            }
        },
       

        "knn": {
            "field": "ml.inference.main_article_vector.predicted_value",
            "query_vector_builder": {
                "text_embedding": {
                    "model_id": ".multilingual-e5-small_linux-x86_64",
                    "model_text": search_term
                }
            },
            "k": k,
            "num_candidates": 10000
        },
        "rank":{"rrf":  {"window_size": k, "annotate_source": "true"}},
        "fields": [
            "id",
            "url",
            "main_article_text",
            "document_image"
        ],
        "_source": "false"
        
    }


BadRequestError: BadRequestError(400, 'x_content_parse_exception', '[1:390] [rrf] unknown field [annotate_source]')
7

Here's another PR with examples for explain.