Slow cosine similarity script

Elastic Stack Elasticsearch
1

Hi,

in a query, I am executing a cosine similarity script. It takes multiple seconds, but the top command shows CPU and memory usage of nearly zero. It looks like a configuration issue to me, but I can not figure out the reason.

I am thinking of increasing the shards size, but maybe there is another reason for the bad performance. If I understood articles correct, the number of shards is okay.
Also having a more specific query seems not promising to me as I am interested in both fields entity and embeddings.

The config is pasted below. What would be a promising change to increase the speed?

Query:

{
   "script_score":{
      "query":{
         "match_all":{}
      },
      "script":{
         "source":"cosineSimilarity(params.query_vector, 'embeddings') + 1.0",
         "params":{
            "query_vector":"[...]"
         }
      }
   }
}

Configuration:

{
   "settings":{
      "number_of_shards":5,
      "number_of_replicas":1
   },
   "mappings":{
      "properties":{
         "id":{
            "type":"keyword"
         },
         "entity":{
            "type":"keyword"
         },
         "embeddings":{
            "type":"dense_vector",
            "dims":200
         }
      }
   }
}

System:

  • CPU: 4x Intel(R) Xeon(R) CPU E5-2695 v3 @ 2.30GHz
  • Memory: 32 GB
  • ulimit: unlimited
  • /proc/sys/vm/swappiness: 1
  • Java heap size: -Xms8g / -Xmx8g
    (increasing to 16g did not really change performance)
  • bootstrap.memory_lock: true
  • index size: 22 million
  • ES version: 7.16

I had a look into, e.g.

2

Update: I did a reindex from 5 to 40 shards (runtime 166m40,433s).
This resulted in a timeout using the new index.
CPU/Memory still not used.
Maybe the speed issue is based on a routing nginx -[socket]-> flask -[python]-> es

3

Update:

  • Used ES API with cURL -> Same runtime (issue not based on nginx/flask)
  • Created stored script and used it in query -> Same runtime

Any ideas how to improve the runtime/performance?

4

cosineSimilarity through script is a slow operation especially if you don't have any restricting filter. In this case the script has to go though all documents and calculate the score. Have you considered using indexed vectors and _knn_search API for ANN? It could be many time faster.

5

Thanks a lot, mayya! I will take a look into kNN search

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.