# Index Pre-Load for Vector Store **URL:** \ **Category:** Elasticsearch\ **Tags:** vector-search\ **Created:** [March 7, 2025, 3:56pm UTC](https://discuss.elastic.co/t/index-pre-load-for-vector-store/375564 "2025-03-07T15:56:29Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![peedeeboy](https://avatars.discourse-cdn.com/v4/letter/p/2acd7d/32.png) [@peedeeboy](https://discuss.elastic.co/u/peedeeboy)\ **Post date:** [March 7, 2025, 3:56pm UTC](https://discuss.elastic.co/t/index-pre-load-for-vector-store/375564/1 "2025-03-07T15:56:29Z")
Hey all 👋 We have dedicated ES clusters we use for our vector database and approx kNN searching (single hnsw\_int8 dense vector) as part of our hybrid-search solution. We've started to push one cluster pretty hard, and noticed before that if we scale a couple more nodes horizontally, performance can get slightly worse in our perf tests under normal load 🤔 My theory is this is probably us not getting the best out of caching with more nodes. I'm aware that vector search relies on the underlying Linux Page Cache and we've already made sure we've got plenty of RAM available on the nodes, so we've started looking into pre-loading the index files [as described here](https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-knn-search.html#dense-vector-preloading) But we don't currently have any `.vex` or `.veq` files in our data folder 😮 What we do have are: - `.si` - `.cfe` / `.cfs` - `.dvd` / `.dvm` - `.fnm` So my question are: - Which of these would we be best to pre-load? - Am I right in saying our vectors are probably stored in the `.cfe` or `.cfs` files (which my Google-fu tells me are Lucene compound files) - If so, what triggers Lucene to make compound files? Is it default now? Is it possible that the index could have either `.vex` and `.veq` and or `.cfe` and `.cfs`? I.e. should we configure pre-load for all those file extensions? Any guidance would be much appreciated! 😃 ---
**Author:** ![Carlos\_D](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carlos_d/32/126245_2.png) [@Carlos\_D](https://discuss.elastic.co/u/Carlos_D)\ **Post date:** [March 10, 2025, 9:53am UTC](https://discuss.elastic.co/t/index-pre-load-for-vector-store/375564/2 "2025-03-10T09:53:05Z")
Hi @peedeeboy : Compound files are normally used when segment sizes is less than 1GB. Is that your case? You could preload compound files, but that's probably too much, more so with quantized vectors - so you won't need to prewarm the non-quantized vector values for example. You can try to [reduce the number of index segments](https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-knn-search.html#_reduce_the_number_of_index_segments) so the compound file format is not used, as segments will be \> 1GB in size. ---
**Author:** ![peedeeboy](https://avatars.discourse-cdn.com/v4/letter/p/2acd7d/32.png) [@peedeeboy](https://discuss.elastic.co/u/peedeeboy)\ **Post date:** [March 10, 2025, 10:04am UTC](https://discuss.elastic.co/t/index-pre-load-for-vector-store/375564/3 "2025-03-10T10:04:57Z")
Thanks @Carlos_D - that's super helpful! 😃 Yes, our segments will be pretty small. We have a trickle of indexing events throughout the day as products come in / out of stock, so lots of small segments getting written then merged by the background process... Even when ES merges segments, we only have ~150K documents max, and because this is a dedicated kNN cluster, those documents are super basic (ID + vector embedding), so the merged segments are _probably_ still pretty small - I have definitely seen `.v*` files in there in the past though! So would I be correct in saying that _theoretically_ when a segment merge happens if the merged segment is \> 1GB, Lucene might decide to write it out as `.vex` + `.veq` instead? If so, I think we'll do a quick perf test with pre-loading `['.cfe', '.cfs', '.vex', '.veq']` and see if makes a difference to us 👍 From what you say, I suspect possibly not... 🤔 ---
**Author:** ![Carlos\_D](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carlos_d/32/126245_2.png) [@Carlos\_D](https://discuss.elastic.co/u/Carlos_D)\ **Post date:** [March 10, 2025, 10:32am UTC](https://discuss.elastic.co/t/index-pre-load-for-vector-store/375564/4 "2025-03-10T10:32:03Z")
> Even when ES merges segments, we only have ~150K documents max At that volume, you might want to experiment with doing [exact kNN search](https://www.elastic.co/guide/en/elasticsearch/reference/8.9/knn-search.html#exact-knn) via `script_score`. You'll get better search results via exact kNN, you'll need to check that the latency is appropriate for your use case. See [this blog post](https://www.elastic.co/search-labs/blog/knn-exact-vs-approximate-search) for more details on approximate vs exact knn. > So would I be correct in saying that _theoretically_ when a segment merge happens if the merged segment is \> 1GB, Lucene might decide to write it out as `.vex` + `.veq` instead? That is correct - the default merge policy for Elasticsearch uses 1GB as the limit for using compound files. > If so, I think we'll do a quick perf test with pre-loading `['.cfe', '.cfs', '.vex', '.veq']` and see if makes a difference to us 👍 From what you say, I suspect possibly not... 🤔 Keep in mind that you're searching over many small segments - that's going to make search slower, as knn needs to go over every segment for getting results. You should try to get less segments for the knn search side - adjusting the merge policy, or doing periodic force\_merge merges could be used for that. ---
**Author:** ![peedeeboy](https://avatars.discourse-cdn.com/v4/letter/p/2acd7d/32.png) [@peedeeboy](https://discuss.elastic.co/u/peedeeboy)\ **Post date:** [March 10, 2025, 10:34am UTC](https://discuss.elastic.co/t/index-pre-load-for-vector-store/375564/5 "2025-03-10T10:34:57Z")
Great stuff - thanks @Carlos_D 👍 that gives us some more things to look into to try and get the best out of our kNN searches 💪 ---
**Author:** ![Carlos\_D](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carlos_d/32/126245_2.png) [@Carlos\_D](https://discuss.elastic.co/u/Carlos_D)\ **Post date:** [March 10, 2025, 10:47am UTC](https://discuss.elastic.co/t/index-pre-load-for-vector-store/375564/6 "2025-03-10T10:47:42Z")
Good luck! Please report back your findings @peedeeboy !