# Can Elasticsearch automatically normalize vectors for dot\_product? **URL:** \ **Category:** Elasticsearch\ **Tags:** vector-search\ **Created:** [December 12, 2023, 1:16pm UTC](https://discuss.elastic.co/t/can-elasticsearch-automatically-normalize-vectors-for-dot-product/349151 "2023-12-12T13:16:45Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![mmaccou](https://avatars.discourse-cdn.com/v4/letter/m/cab0a1/32.png) [@mmaccou](https://discuss.elastic.co/u/mmaccou)\ **Post date:** [December 12, 2023, 1:16pm UTC](https://discuss.elastic.co/t/can-elasticsearch-automatically-normalize-vectors-for-dot-product/349151/1 "2023-12-12T13:16:45Z")
Can Elasticsearch automatically normalize vectors for dot\_product? I haven't found it the docs yet, but wanted to double check here before I manually add the step in my code. ---
**Author:** ![BenTrent](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bentrent/32/33915_2.png) [@BenTrent](https://discuss.elastic.co/u/BenTrent)\ **Post date:** [December 12, 2023, 1:35pm UTC](https://discuss.elastic.co/t/can-elasticsearch-automatically-normalize-vectors-for-dot-product/349151/2 "2023-12-12T13:35:41Z")
Hey @mmaccou , We do not auto-normalize the vectors. In 8.12, cosine will be updated so that its performance will be improved like dot-product for typical usage: [https://github.com/elastic/elasticsearch/pull/99445](https://github.com/elastic/elasticsearch/pull/99445) ---
**Author:** ![mmaccou](https://avatars.discourse-cdn.com/v4/letter/m/cab0a1/32.png) [@mmaccou](https://discuss.elastic.co/u/mmaccou)\ **Post date:** [December 12, 2023, 1:50pm UTC](https://discuss.elastic.co/t/can-elasticsearch-automatically-normalize-vectors-for-dot-product/349151/3 "2023-12-12T13:50:50Z")
Oh perfect! Is there a release date for 8.12? Will the performance match dot\_product, or will it just be improved, but not quite matched? Also, a follow-up question since this is new to me, when I bulk index content into Elasticsearch using an inference pipeline with dot\_product similarity, is there a way to normalize the vector first? It's not clear how that would even be possible using the inference pipeline. ---
**Author:** ![Serena\_Chou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/serena_chou/32/100087_2.png) [@Serena\_Chou](https://discuss.elastic.co/u/Serena_Chou)\ **Post date:** [December 12, 2023, 4:17pm UTC](https://discuss.elastic.co/t/can-elasticsearch-automatically-normalize-vectors-for-dot-product/349151/4 "2023-12-12T16:17:49Z")
Hey there , we can't share exact release dates for this exciting functionality yet - but rest assured it is actively in the works and will be available soon. I suggest that for more updates you can subscribe to our more technical site with your RSS feed so you can see the stuff that will be trickling into 8.12 - [All — Elastic Search Labs](https://www.elastic.co/search-labs/blog/categories) for instance the fastest Apache Lucene release ever. [Apache Lucene 9.9, the fastest Lucene release ever — Elastic Search Labs](https://www.elastic.co/search-labs/blog/articles/apache-lucene-9.9-search-speedups) ---
**Author:** ![mmaccou](https://avatars.discourse-cdn.com/v4/letter/m/cab0a1/32.png) [@mmaccou](https://discuss.elastic.co/u/mmaccou)\ **Post date:** [December 12, 2023, 4:40pm UTC](https://discuss.elastic.co/t/can-elasticsearch-automatically-normalize-vectors-for-dot-product/349151/5 "2023-12-12T16:40:11Z")
Hi Serena. Thanks for directing me to the other site. Can you also take a look at my follow up question? I'm trying to figure out the right course of action until 8.12 is released. 1. When I bulk index content into Elasticsearch using an inference pipeline with dot\_product similarity, is there a way to normalize the vector first? It's not clear how that would even be possible using the inference pipeline. 2. Once 8.12 is released, will I have to re-index everything to take advantage of the the cosine updates? ---
**Author:** ![Serena\_Chou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/serena_chou/32/100087_2.png) [@Serena\_Chou](https://discuss.elastic.co/u/Serena_Chou)\ **Post date:** [December 14, 2023, 11:09pm UTC](https://discuss.elastic.co/t/can-elasticsearch-automatically-normalize-vectors-for-dot-product/349151/6 "2023-12-14T23:09:50Z")
Yeah inference processor won't do it for you. Our great SAs at Elastic actually have a few examples that I stole here: > > > import numpy as np > > > > > > def normalize\_vector(vector): > > > ... magnitude = np.linalg.norm(vector) > > > ... if magnitude == 0: > > > ... return vector > > > ... normalized\_vector = vector / magnitude > > > ... return normalized\_vector > > > ... > > > > > > # test > > > > > > vector = np.array([3, 4, 5]) > > > normalized\_vector = normalize\_vector(vector) > > > print("Original Vector:", vector) > > > Original Vector: [3 4 5] > > > print("Normalized Vector:", normalized\_vector) > > > Normalized Vector: [0.42426407 0.56568542 0.70710678] ---
**Author:** ![mmaccou](https://avatars.discourse-cdn.com/v4/letter/m/cab0a1/32.png) [@mmaccou](https://discuss.elastic.co/u/mmaccou)\ **Post date:** [December 15, 2023, 12:51am UTC](https://discuss.elastic.co/t/can-elasticsearch-automatically-normalize-vectors-for-dot-product/349151/7 "2023-12-15T00:51:02Z")
Thank you for sharing this example! I appreciate it. This addresses the first question, right? For the second, when 8.12 is released and cosine similarity is improved, will I need to reindex all of my data? ---
**Author:** ![Serena\_Chou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/serena_chou/32/100087_2.png) [@Serena\_Chou](https://discuss.elastic.co/u/Serena_Chou)\ **Post date:** [December 19, 2023, 10:01pm UTC](https://discuss.elastic.co/t/can-elasticsearch-automatically-normalize-vectors-for-dot-product/349151/8 "2023-12-19T22:01:28Z")
Yes, that code is Python btw, and should get you started for normalizing vectors. You aren't able to update an existing index's similarity setting so you'd want to pick which one you want to go with cosine or dot\_product. If you'd like to switch to cosine later, you'd have to index to a new index with your chosen similarity setting. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 16, 2024, 10:01pm UTC](https://discuss.elastic.co/t/can-elasticsearch-automatically-normalize-vectors-for-dot-product/349151/9 "2024-01-16T22:01:42Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.