Searching with Dense Vector

Elastic Stack Elasticsearch
1

Is there a way to apply a query directly to DenseVector e.g. Nearest Neighbors. I understand that a vector field (i.e. Dense or Sparse) is accessible via scoring. But I'm rather curious about true sense vector search like provided via FAISS i.e. is it possible to send query like find me n number of nearest neighbors to a certain vector of same dimension?

2

Hello,
the way to find n number of nearest neighbors is through scoring.
Scoring functions let you define what is "nearest" to you.

For example, the following query finds top 5 nearest documents, where nearest is defined as inversely proportional to euclidean distance.

GET my_index/_search
{
  "size" : 5,
  "query": {
    "script_score": {
      "query" : {
        "match_all" : {}
      },
      "script": {
        "source": "1 / (1 + l2norm(params.queryVector, doc['my_dense_vector']))",
        "params": {
          "queryVector": [4, 3.4, -0.2]
        }
      }
    }
  }
}
3

Thanks a lot for the response. However the score will be calculated for all doc['my_dense_vector'] in this query, so performance might be hit. Am I correct assuming that?

4

Correct, score will be calculated by all documents matching the query. You can provide a more restrictive query to limit the number of matching documents.

And you are right, calculating score for all documents can be slow. We are working on ways to do approximate knn search.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.