What is the best way to search one index with keyword and another index with vector and combine the search results?

[zli]

Hi there,

I'm currently working on a project where I need to perform searches across multiple indices in Elasticsearch and combine the results into a single ranked list. I have one index where keyword search is performed and another index where vector search only is used. For the large index, it would be good to keep it as keyword search because the index is pretty large and updated frequently. For the latter one, I'd like to use vector search only because based on some experiments results, using vector search on this index gives a better search accuracy.

I wonder what is the best way to do this in a single call to both indices, using keyword search on index 1, vector search on index 2, and have a single sorted list of hits in response.

I've tried to use the Multi-Search API to execute searches on both indices in a single request, and I've received separate responses containing the results from each index. However, the keyword search is not normalized so the score can be sometime as large as 40+, while the vector search is normalized to [0,1]. Therefore, I'm not sure how to merge the two responses and re-rank them.

Any other solution is also welcome! Thanks a lot!

[Christian_Dahlqvist]

@zli

Therefore, I'm not sure how to merge the two responses and re-rank them.

If you are not sure how you would merge two disparate result sets that have little in common, how would Elasticsearch correctly be able to do so?

I would recommend running 2 queries using the multi serach API and combine the results in your application as I doubt there is any way to normalise or "correctly" combine the result sets.