Hi all,
I have a specific question on whether we need all the vectors to be persisted in memory on the Elasticsearch ML node (or in the other ES nodes) when running ML search.
We're building an application at my end which needs to implement hybrid search on ES for millions of records of data. We compute embeddings for some of the data fields in each record and use this as part of the hybrid search feature (which uses BM25 and vector similarity via RRF).
Want to understand whether these embeddings need to be persisted in memory, or whether they can be persisted to indexes, like with a vector database.