We’ve been using ES 8.9.0 and are looking to upgrade, so I started testing performance with 9.1.3 and found it to be very slow. So, I went back to 8.19.3 and found that it’s performance is fine. But we’d like to not be stuck on ES 8, so I’m wondering if there’s anything we can do to fix the performance.
Our use case involves getting the cosine sim for multiple vectors and feeding the those values into a non-linear model to generate a final score. Because of this, we can’t use kNN so we set the “index”: false for the dense_vectors.
To simplify the test, I created an index with just two vectors fields:
{
"mappings": {
"properties": {
"vectorOne": {
"type": "dense_vector",
"dims": 768,
"index": false
},
"vectorTwo": {
"type": "dense_vector",
"dims": 768,
"index": false
}
}
},
"settings": {
"index": {
"mode": "standard",
"number_of_shards": "4",
"number_of_replicas": "0"
}
}
}
Then I loaded 250k documents with the two vectors and used the following query for performance testing:
{
"query": {
"script_score": {
"query": {
"match_all": {}
},
"script": {
"source": "Math.max(0, cosineSimilarity(params.vectorOne, 'vectorOne') + cosineSimilarity(params.vectorTwo, 'vectorTwo'))",
"params": {
"vectorOne": [...],
"vectorTwo": [...]
}
}
}
}
}
In 8.19.3, this query averages 248 ms. In 9.1.3, it averages 683 ms. This is running on a cluster with a single AWS r7a.xlarge data node, 4 GB JVM heap (~28 GB available for disk cache). Are there any settings that could improve the performance? I tried “mode”: “standard” to see if switching away from LogDB would help, but it didn’t seem to make a difference. I tried adding more shards, but that made performance worse. I tried adding -Dvector.rescoring.directio=false in case it was using DirectIO instead of going through the disk cache, but that had no effect.