# Slow cosine similarity script

**URL:** <https://discuss.elastic.co/t/slow-cosine-similarity-script/299496>\
**Category:** Elasticsearch\
**Created:** [March 11, 2022, 7:50pm UTC](https://discuss.elastic.co/t/slow-cosine-similarity-script/299496 "2022-03-11T19:50:21Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![adibaba](https://avatars.discourse-cdn.com/v4/letter/a/2acd7d/32.png) [@adibaba](https://discuss.elastic.co/u/adibaba)\
**Post date:** [March 11, 2022, 7:50pm UTC](https://discuss.elastic.co/t/slow-cosine-similarity-script/299496/1 "2022-03-11T19:50:22Z")

</div>

Hi,

in a query, I am executing a **cosine similarity script**. It takes multiple seconds, but the `top` command shows **CPU and memory usage of nearly zero**. It looks like a configuration issue to me, but I can not figure out the reason.

I am thinking of increasing the shards size, but maybe there is another reason for the bad performance. If I understood articles correct, the number of shards is okay.  
Also having a more specific query seems not promising to me as I am interested in both fields `entity` and `embeddings`.

The config is pasted below. What would be a promising change to increase the speed?

**Query:**

```auto
{
   "script_score":{
      "query":{
         "match_all":{}
      },
      "script":{
         "source":"cosineSimilarity(params.query_vector, 'embeddings') + 1.0",
         "params":{
            "query_vector":"[...]"
         }
      }
   }
}

```

**Configuration:**

```auto
{
   "settings":{
      "number_of_shards":5,
      "number_of_replicas":1
   },
   "mappings":{
      "properties":{
         "id":{
            "type":"keyword"
         },
         "entity":{
            "type":"keyword"
         },
         "embeddings":{
            "type":"dense_vector",
            "dims":200
         }
      }
   }
}

```

**System:**

- CPU: 4x Intel(R) Xeon(R) CPU E5-2695 v3 @ 2.30GHz
- Memory: 32 GB
- ulimit: unlimited
- /proc/sys/vm/swappiness: 1
- Java heap size: -Xms8g / -Xmx8g  
(increasing to 16g did not really change performance)
- bootstrap.memory\_lock: true
- index size: **22 million**
- ES version: **7.16**

I had a look into, e.g.

- [Text similarity search in Elasticsearch using vector fields | Elastic Blog](https://www.elastic.co/blog/text-similarity-search-with-vectors-in-elasticsearch)
- [How vector based text similarity works under the hood?](https://discuss.elastic.co/t/how-vector-based-text-similarity-works-under-the-hood/237498)
- [Prototype: Product quantization for nn search by mayya-sharipova · Pull Request #51243 · elastic/elasticsearch · GitHub](https://github.com/elastic/elasticsearch/pull/51243#issuecomment-576454016)
- [Size your shards | Elasticsearch Guide [7.16] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/7.16/size-your-shards.html)
- [How many shards should I have in my Elasticsearch cluster? | Elastic Blog](https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster)

---

<div class="post-metadata">

**Author:** ![adibaba](https://avatars.discourse-cdn.com/v4/letter/a/2acd7d/32.png) [@adibaba](https://discuss.elastic.co/u/adibaba)\
**Post date:** [March 12, 2022, 8:20am UTC](https://discuss.elastic.co/t/slow-cosine-similarity-script/299496/2 "2022-03-12T08:20:57Z")

</div>

Update: I did a reindex from 5 to **40 shards** (runtime 166m40,433s).  
This resulted in a timeout using the new index.  
CPU/Memory still not used.  
Maybe the speed issue is based on a routing nginx -[socket]-\> flask -[python]-\> es

---

<div class="post-metadata">

**Author:** ![adibaba](https://avatars.discourse-cdn.com/v4/letter/a/2acd7d/32.png) [@adibaba](https://discuss.elastic.co/u/adibaba)\
**Post date:** [March 13, 2022, 9:36am UTC](https://discuss.elastic.co/t/slow-cosine-similarity-script/299496/3 "2022-03-13T09:36:48Z")

</div>

Update:

- Used **ES API** with **cURL** -\> Same runtime (issue not based on nginx/flask)
- Created **stored script** and used it in query -\> Same runtime

Any ideas how to improve the runtime/performance?

---

<div class="post-metadata">

**Author:** ![mayya](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mayya/32/83147_2.png) [@mayya](https://discuss.elastic.co/u/mayya)\
**Post date:** [April 1, 2022, 10:34pm UTC](https://discuss.elastic.co/t/slow-cosine-similarity-script/299496/4 "2022-04-01T22:34:46Z")

</div>

`cosineSimilarity` through script is a slow operation especially if you don't have any restricting filter. In this case the script has to go though all documents and calculate the score. Have you considered using indexed vectors and [\_knn\_search API](https://www.elastic.co/guide/en/elasticsearch/reference/master/knn-search.html) for ANN? It could be many time faster.

---

<div class="post-metadata">

**Author:** ![adibaba](https://avatars.discourse-cdn.com/v4/letter/a/2acd7d/32.png) [@adibaba](https://discuss.elastic.co/u/adibaba)\
**Post date:** [April 1, 2022, 11:02pm UTC](https://discuss.elastic.co/t/slow-cosine-similarity-script/299496/5 "2022-04-01T23:02:22Z")

</div>

Thanks a lot, mayya! I will take a look into kNN search

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 29, 2022, 11:02pm UTC](https://discuss.elastic.co/t/slow-cosine-similarity-script/299496/6 "2022-04-29T23:02:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.