Support for colpali and other late interaction based retrieval

Elastic Stack Elasticsearch
1

Does elasticsearch support search based on Embeddings from models like colqwen, colpali, etc.?

If yes, where can I find supporting documentation?

2

Hi @dbasu , welcome to our community.

Elasticsearch supports the dense_vector field, which allows storing embeddings from any model and enables semantic operations, making it function as a vector database.

Here are some similar examples:

1 Like
3

Thanks Alex. With the embeddings from colpali, the similarity computation is a bit more involved.

I believe its referred to as late interaction and involves a maximum similarity over multiple cosine similarities just to compute the number for each pair of query, document.

Wondering if this is also supported.

I'm unable to share links here but you can try to look up colpali, colqwen and colbert

4

Hey @dbasu:

Version 8.18 will include the new rank_vectors field type that will allow late interaction similarity.

At this time 8.18 has not yet been released, but is available on serverless in case you want to test it out.

You can check the upcoming feature here.

Stay tuned for 8.18!