Is my vector search quick enough

Elastic Stack Elasticsearch
1

I have 1536 dimension vectors,
During benchmarking I am doing vector similarity search using script (not knn) by matching one vector agains 30k stored documents with vectors and getting response in about 4s
The time increases to 10s for 100k documents with vectors.

Is this slow, because I have 32 cores and 128gb ram

given below my query script

"must": [
                    {
                        "script_score": {
                            "query": { "match_all": {} },
                            "script": {
                                "source": """
                                    if (doc['vector'].size() > 0) {
                                        return cosineSimilarity(params.query_vector, 'vector') + 1.0;
                                    } else {
                                        return 0.0;
                                    }
                                """,
                                "params": { "query_vector": vector }
                            }
                        }
                    }
                ]
2

That's expected as it needs to compute the cosine for every single vector (brute-force). From 30k to 100k means much more vectors.
Using knn should be much faster.

Also look at Script score query | Elasticsearch Guide [8.17] | Elastic

During vector functions' calculation, all matched documents are linearly scanned. Thus, expect the query time grow linearly with the number of matched documents. For this reason, we recommend to limit the number of matched documents with a query parameter.

3

If your index is small and with a limited number of shards and segments I suspect Elasticsearch will only be able to utilise a few cores at most as there as far as I know are limits to parallelism.

4

Thanks @dadoonet for responding.
So for 30k documents 4s is expected time?
I think it's a lot, I am going to have 5 million documents so will script vector search not remain an option for me?

5

Thanks @Christian_Dahlqvist for replying
I have a big index of about 5 million vectors with 5 shards, I have preloaded vex and veq, i.e. hnsw graph and quantized vectors.
Given below is my index setting
"store": {
"preload": [
"vex",
"veq"
]
},

anything else I need to do ?

6

If you want to search always on the 5m documents using a brute force method without any filter as recommended in the document I linked, then I suspect that it will be super slow. May be after some queries it will become faster if all the files are loaded in RAM but still this will take time as specified in the first doc I linked.

My advice is to switch to kNN query. That's what I'm using myself and it's extremely powerful.

1 Like