Managed Elastic search for billion scale dense vector index and performance

[cvgoudar]

We want to understand on the requirements and expected response time for vector based search with close to 1 Billion semantic vector index. The dense vector is 768 dimensional.

What would be the appropriate configuration of Managed Elastic search to get response with knn search within 100-200 ms response time. We would like to get close to Top 20 matches with vector similarity.

Also will there be performance impact if filter option is used

[Julie_Tibshirani]

Hello @cvgoudar, unfortunately we don't have published benchmarks for this configuration (1 billion vectors with 768 dimensions). To determine if the performance will be acceptable, our recommendation is to test your dataset + queries using a benchmarking framework like Elasticsearch rally (https://esrally.readthedocs.io). You can start with a single node, fitting as many vectors as possible into it, and then calculate how many nodes you will need for the full 1 billion vector dataset. Here are some resources that can help:

• The kNN search tuning guide: Tune approximate kNN search | Elasticsearch Guide [8.5] | Elastic. This guide explains that memory is a primary bottleneck for vector search -- you need to have enough RAM available on the node to hold all the vector data in page cache.
• In an upcoming release, we'll add a support for lower-precision vector element types: https://github.com/elastic/elasticsearch/pull/90774. This is sometimes called "quantization" and can really help reduce memory requirements for large datasets like yours.

About filtering: yes, approximate kNN search is usually slower when using a 'filter'. This is because the search needs to skip over documents that do not match the filter. If a filter is very selective (meaning it matches few documents), the performance impact can be greater. If you are always using a filter and it's quite selective, then you should check whether exact kNN search (k-nearest neighbor (kNN) search | Elasticsearch Guide [8.5] | Elastic) is a better fit for your use case.

[ruslaniv]

I'd be VERY cautious and run extensive testing before indexing this large of a dataset. Check this discussion: Dense vector field space requirements