Implement my own Hybrid Search

Hi team,

I've developed a hybrid search algorithm. Initially, I perform a BM25 search, obtaining the top k results (id, score). Subsequently, a k-nearest neighbors (KNN) search is executed, yielding another set of top k results (id, score). These results from BM25 and KNN are then combined using a custom algorithm. Finally, the combined results are sorted, and the top-k items are retrieved by calling the search function to obtain the original text by id.

While the current implementation involves three calls to the search function and may seem a bit complex, I'm exploring ways to simplify it like linear combination or relevance ranking fusion (RRF) in Elasticsearch. This could enhance the efficiency and readability of the hybrid search algorithm.

Please let me know if there are any suggestions or improvements you would like to discuss.

Hi there!

We have a number of resources to help with hybrid search. You can use RRF rather than doing this score balancing client side.

see here Hybrid Search — Elastic Search Labs or our notebook examples here

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.