We recently migrated our Elasticsearch clusters from version 7.10.2 to 8.13.0 and have noticed an increase in search latency.
Our shard configuration, cluster, and index settings remain unchanged.
To investigate the cause of this issue, we conducted query profiling for specific queries. Interestingly, we observed that the cumulative shard time is slightly higher than the query time in ES 8. In ES 7, these times were identical. You can see this difference in the attached screenshot. (3.1 ms vs 3.9 ms) We suspect this discrepancy might be the reason for the increased latency.
It's hard to say from the screenshot. Can you share the profile output?
Keep in mind however that profile timings are an approximation - they are based on sampling. They are useful to compare some part of the queries to others, and understand where most of the time is being spent.
It would be useful to compare profile outputs from v7 to v8 to understand whether the query has been rewritten differently, or some aspect of the query execution is now taking longer than in v7, and work from there.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
