We want to understand on the requirements and expected response time for vector based search with close to 1 Billion semantic vector index. The dense vector is 768 dimensional.
What would be the appropriate configuration of Managed Elastic search to get response with knn search within 100-200 ms response time. We would like to get close to Top 20 matches with vector similarity.
Also will there be performance impact if filter option is used
Hello @cvgoudar, unfortunately we don't have published benchmarks for this configuration (1 billion vectors with 768 dimensions). To determine if the performance will be acceptable, our recommendation is to test your dataset + queries using a benchmarking framework like Elasticsearch rally (https://esrally.readthedocs.io). You can start with a single node, fitting as many vectors as possible into it, and then calculate how many nodes you will need for the full 1 billion vector dataset. Here are some resources that can help:
In an upcoming release, we'll add a support for lower-precision vector element types: https://github.com/elastic/elasticsearch/pull/90774. This is sometimes called "quantization" and can really help reduce memory requirements for large datasets like yours.
About filtering: yes, approximate kNN search is usually slower when using a 'filter'. This is because the search needs to skip over documents that do not match the filter. If a filter is very selective (meaning it matches few documents), the performance impact can be greater. If you are always using a filter and it's quite selective, then you should check whether exact kNN search (k-nearest neighbor (kNN) search | Elasticsearch Guide [8.5] | Elastic) is a better fit for your use case.