Pros and Cons of using Elastic as a vector database?

Elastic Stack Elasticsearch
1

I'm comparing Elastic vs other pure vector databases vs Mongodb/redis offerings. Is anything wrong or supplemental? Thank you!

Pros:

  • It's an Elastic product, meaning high SLA and needless to buy other products when doing business with Elastic.
  • Have built-in Embedding models: ELSER.
  • Hybrid search with text+vector
  • Security

Cons:

  • Doesn't support quantization, which may be too exhaustive when storing vast numbers of vectors.
  • Performance seems poor from ann-benchmarks
  • Does not support CUDA/GPU, versus milvus support Nvidia RAPIDS
2

Is it worth mentioning that it only supports hnsw as the knn algorithm?

Also only supports dot product, L2norm and cosine distance algorithms. ie: no hamming distance, unlike milvus, OpenSearch.

Doesn't it basically operate FAISS under the hood, same as Milvus?

I've done a few performance tests using elasticsearch as a vector database vs milvus vs opensearch vs ...

Performance-wise, elasticsearch using knn is comparable to Milvus in my experience, but there are downsides:

  1. It eats up alllll the resources (maxed out CPU and ate up loads of RAM) whereas milvus was more subtle
  2. the limited distance algorithms meant I couldn't use it, in the end (I was looking to do hamming distance on PDQs)

However, Milvus is overengineered, IMHO, especially when you compare it to something like qdrant, for example. The fact that milvus "standalone" still requires etcd and minio as separate services says enough there.

In cases where the vector is associated with other document data (eg: an image vector and here are the image details, height, width, exif or whatever), I'd way rather choose elasticsearch to maintain a simple stack.

These are all things that benchmarks just don't cover. They tell you how fast they are and how good the F-measure is. But they don't tell you how much resource they used and how easy they are to deploy and maintain etc. You have to figure those out for yourself!

1 Like
3

OpenSearch/OpenDistro are AWS run products and differ from the original Elasticsearch and Kibana products that Elastic builds and maintains. You may need to contact them directly for further assistance.

(This is an automated response from your friendly Elastic bot. Please report this post if you have any suggestions or concerns :elasticheart: )

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.