# Support for colpali and other late interaction based retrieval

**URL:** <https://discuss.elastic.co/t/support-for-colpali-and-other-late-interaction-based-retrieval/373864>\
**Category:** Elasticsearch\
**Created:** [January 30, 2025, 12:01am UTC](https://discuss.elastic.co/t/support-for-colpali-and-other-late-interaction-based-retrieval/373864 "2025-01-30T00:01:28Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![dbasu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dbasu/32/141034_2.png) [@dbasu](https://discuss.elastic.co/u/dbasu)\
**Post date:** [January 30, 2025, 12:01am UTC](https://discuss.elastic.co/t/support-for-colpali-and-other-late-interaction-based-retrieval/373864/1 "2025-01-30T00:01:28Z")

</div>

Does elasticsearch support search based on Embeddings from models like colqwen, colpali, etc.?

If yes, where can I find supporting documentation?

---

<div class="post-metadata">

**Author:** ![Alex\_Salgado-Elastic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alex_salgado-elastic/32/103081_2.png) [@Alex\_Salgado-Elastic](https://discuss.elastic.co/u/Alex_Salgado-Elastic)\
**Post date:** [January 31, 2025, 6:28pm UTC](https://discuss.elastic.co/t/support-for-colpali-and-other-late-interaction-based-retrieval/373864/2 "2025-01-31T18:28:19Z")

</div>

Hi @dbasu , welcome to our community.

Elasticsearch supports the **dense\_vector** field, which allows storing embeddings from any model and enables semantic operations, making it function as a vector database.

Here are some similar examples:

- [Alternative Approach for Parsing PDFs in RAG](https://www.elastic.co/search-labs/blog/alternative-approach-for-parsing-pdfs-in-rag)
- [Build RAG in Elastic 8.15](https://www.elastic.co/search-labs/blog/build-rag-in-elastic-815)
- [Chat with PDF - Elastic Playground](https://www.elastic.co/search-labs/blog/chat-with-pdf-elastic-playground)

---

<div class="post-metadata">

**Author:** ![dbasu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dbasu/32/141034_2.png) [@dbasu](https://discuss.elastic.co/u/dbasu)\
**Post date:** [January 31, 2025, 7:11pm UTC](https://discuss.elastic.co/t/support-for-colpali-and-other-late-interaction-based-retrieval/373864/3 "2025-01-31T19:11:07Z")

</div>

Thanks Alex. With the embeddings from colpali, the similarity computation is a bit more involved.

I believe its referred to as late interaction and involves a maximum similarity over multiple cosine similarities just to compute the number for each pair of query, document.

Wondering if this is also supported.

I'm unable to share links here but you can try to look up colpali, colqwen and colbert

---

<div class="post-metadata">

**Author:** ![Carlos\_D](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carlos_d/32/126245_2.png) [@Carlos\_D](https://discuss.elastic.co/u/Carlos_D)\
**Post date:** [February 3, 2025, 6:51pm UTC](https://discuss.elastic.co/t/support-for-colpali-and-other-late-interaction-based-retrieval/373864/4 "2025-02-03T18:51:41Z")

</div>

Hey @dbasu:

Version 8.18 will include the new `rank_vectors` field type that will allow late interaction similarity.

At this time 8.18 has not yet been released, but is available on [serverless](https://www.elastic.co/guide/en/serverless/current/intro.html) in case you want to test it out.

You can check the upcoming feature [here](https://www.elastic.co/guide/en/elasticsearch/reference/8.18/rank-vectors.html).

Stay tuned for 8.18!

---

<div class="post-metadata">

**Author:** ![dbasu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dbasu/32/141034_2.png) [@dbasu](https://discuss.elastic.co/u/dbasu)\
**Post date:** [March 5, 2025, 7:57pm UTC](https://discuss.elastic.co/t/support-for-colpali-and-other-late-interaction-based-retrieval/373864/5 "2025-03-05T19:57:55Z")

</div>

Hi @Carlos_D , I think the link you shared about the upcoming feature is inactive.

---

<div class="post-metadata">

**Author:** ![Carlos\_D](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carlos_d/32/126245_2.png) [@Carlos\_D](https://discuss.elastic.co/u/Carlos_D)\
**Post date:** [March 6, 2025, 6:45am UTC](https://discuss.elastic.co/t/support-for-colpali-and-other-late-interaction-based-retrieval/373864/6 "2025-03-06T06:45:39Z")

</div>

Sorry @dbasu , some doc changes have been ongoing. You can see `rank_vectors` docs for the upcoming 8.18 release [here](https://www.elastic.co/guide/en/elasticsearch/reference/8.18/rank-vectors.html).

---

<div class="post-metadata">

**Author:** ![Carlos\_D](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carlos_d/32/126245_2.png) [@Carlos\_D](https://discuss.elastic.co/u/Carlos_D)\
**Post date:** [March 18, 2025, 3:03pm UTC](https://discuss.elastic.co/t/support-for-colpali-and-other-late-interaction-based-retrieval/373864/7 "2025-03-18T15:03:24Z")

</div>

Right off the press, @dbasu !

> **[ColPali & Elasticsearch: Searching complex documents - Elasticsearch Labs](https://www.elastic.co/search-labs/blog/elastiacsearch-colpali-document-search)**
>
> Learn about ColPali and explore how to use it to search through complex documents in Elasticsearch, including tables, figures, multiple columns & more.
