We are upgrading from ES 5.6.3 to ES 7.8.0. As part of this, we are doing performance tests to see
if there is an improvement or actual degradation in performance.
For this we've setup 2 identical clusters in AWS, each cluster containing the index (3 shards) containing 9M docs.
Queries performed (with scroll of '1m'):
- Date/time range ( 5.5M+ or 3.4M hits depending on provided range)
- Date/time range + terms (1.2M+ hits or 750K+ hits depending on provided range)
We collected timings using the profile api + timed the ES calls from our C# application.
What we were seeing is that the same queries against ES 7 take about 16% longer. This contradicts the timings we see in the profile data, which shows better times when compared against ES 5 profile data.
So question is: What further action can we take to investigate this performance degradation?
Am able to provide additional info as required/requested.