Slow cosine similarity script

[adibaba (Adrian)]

Hi,

in a query, I am executing a cosine similarity script. It takes multiple seconds, but the top command shows CPU and memory usage of nearly zero. It looks like a configuration issue to me, but I can not figure out the reason.

I am thinking of increasing the shards size, but maybe there is another reason for the bad performance. If I understood articles correct, the number of shards is okay.
Also having a more specific query seems not promising to me as I am interested in both fields entity and embeddings.

The config is pasted below. What would be a promising change to increase the speed?

Query:

{
   "script_score":{
      "query":{
         "match_all":{}
      },
      "script":{
         "source":"cosineSimilarity(params.query_vector, 'embeddings') + 1.0",
         "params":{
            "query_vector":"[...]"
         }
      }
   }
}

Configuration:

{
   "settings":{
      "number_of_shards":5,
      "number_of_replicas":1
   },
   "mappings":{
      "properties":{
         "id":{
            "type":"keyword"
         },
         "entity":{
            "type":"keyword"
         },
         "embeddings":{
            "type":"dense_vector",
            "dims":200
         }
      }
   }
}

System:

• CPU: 4x Intel(R) Xeon(R) CPU E5-2695 v3 @ 2.30GHz
• Memory: 32 GB
• ulimit: unlimited
• /proc/sys/vm/swappiness: 1
• Java heap size: -Xms8g / -Xmx8g
  (increasing to 16g did not really change performance)
• bootstrap.memory_lock: true
• index size: 22 million
• ES version: 7.16

I had a look into, e.g.

• Text similarity search in Elasticsearch using vector fields | Elastic Blog
• How vector based text similarity works under the hood?
• Prototype: Product quantization for nn search by mayya-sharipova · Pull Request #51243 · elastic/elasticsearch · GitHub
• Size your shards | Elasticsearch Guide [7.16] | Elastic
• How many shards should I have in my Elasticsearch cluster? | Elastic Blog

[adibaba (Adrian)]

Update: I did a reindex from 5 to 40 shards (runtime 166m40,433s).
This resulted in a timeout using the new index.
CPU/Memory still not used.
Maybe the speed issue is based on a routing nginx -[socket]-> flask -[python]-> es

[adibaba (Adrian)]

Update:

• Used ES API with cURL -> Same runtime (issue not based on nginx/flask)
• Created stored script and used it in query -> Same runtime

Any ideas how to improve the runtime/performance?

[mayya (Mayya Sharipova)]

cosineSimilarity through script is a slow operation especially if you don't have any restricting filter. In this case the script has to go though all documents and calculate the score. Have you considered using indexed vectors and _knn_search API for ANN? It could be many time faster.

[adibaba (Adrian)]

Thanks a lot, mayya! I will take a look into kNN search