# Vector Search in ElasticSearch + Filtering using other fields

**URL:** <https://discuss.elastic.co/t/vector-search-in-elasticsearch-filtering-using-other-fields/356032>\
**Category:** Elasticsearch\
**Tags:** vector-search\
**Created:** [March 23, 2024, 12:54pm UTC](https://discuss.elastic.co/t/vector-search-in-elasticsearch-filtering-using-other-fields/356032 "2024-03-23T12:54:59Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![anuj99](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anuj99/32/133376_2.png) [@anuj99](https://discuss.elastic.co/u/anuj99)\
**Post date:** [March 23, 2024, 12:54pm UTC](https://discuss.elastic.co/t/vector-search-in-elasticsearch-filtering-using-other-fields/356032/1 "2024-03-23T12:54:59Z")

</div>

Following is the use case:  
I am storing vectors and meta data for a document in a single index. However, if there is any change in the vector, I also need to fetch the meta data and then update the document in the index . This is also true vice vera.

If i go for partial updates , this can result in data inconsistency considering changes in vector and also meta data updates events are ingested at high frequency.

Is there any way to store the two data in different indexes and use a common value while filtering based on some common id. First by meta data and then vector search based on the filtered metadata in a single query.

I don't want to execute two queries in a two step process as it is not optimal for large datasets with millions of documents

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 23, 2024, 4:07pm UTC](https://discuss.elastic.co/t/vector-search-in-elasticsearch-filtering-using-other-fields/356032/2 "2024-03-23T16:07:01Z")

</div>

I am not sure I understand the update logic and constraints you are describing, so some clarification woiuld be useful.

> [@anuj99](#):
>
> Is there any way to store the two data in different indexes and use a common value while filtering based on some common id.

This is basically a join operation, which Elasticsearch does not support. If you want to store the vector and metadata in separate documents so you can update them individually you might want to test using [the join data type](https://www.elastic.co/guide/en/elasticsearch/reference/8.12/parent-join.html) to create a parent/child relationship within a single index. This does add overhead and make queries more complex and expensive but may be an option. I am not sure this will work but it is the only option I can think of.

---

<div class="post-metadata">

**Author:** ![anuj99](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anuj99/32/133376_2.png) [@anuj99](https://discuss.elastic.co/u/anuj99)\
**Post date:** [April 9, 2024, 6:15am UTC](https://discuss.elastic.co/t/vector-search-in-elasticsearch-filtering-using-other-fields/356032/3 "2024-04-09T06:15:56Z")

</div>

Thanks got the quick response.

What about terms lookup with 2 indexes.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [April 9, 2024, 6:41am UTC](https://discuss.elastic.co/t/vector-search-in-elasticsearch-filtering-using-other-fields/356032/4 "2024-04-09T06:41:54Z")

</div>

> [@anuj99](#):
>
> What about terms lookup with 2 indexes.

What do you mean? Can you please elaborate?

As I stated earlier, Elasticsearch does not support joins across indices.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 7, 2024, 6:42am UTC](https://discuss.elastic.co/t/vector-search-in-elasticsearch-filtering-using-other-fields/356032/5 "2024-05-07T06:42:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
