ES 5.2 query performance much worse than ES 2.4?

Elastic Stack Elasticsearch
1

I just upgraded our existing ES 2.4 cluster to ES 5.2 with following steps:

  1. Take snapshot of all indices on ES 2.4 cluster
  2. Remove the ES 2.4 cluster
  3. Create new ES 5.2 cluster with same hardware configurations
  4. Restore the last taken snapshot to new created cluster
  5. Run load test

I found that the query performance on ES 5.2 is much worse then on old ES 2.4 cluster.

On old ES 2.4, the average response time is around 50 ms, now it is 240 ms.

Is this a known perf issue?

2

We've seen similar degradation. Here the major slowdown was caused by parent-child relationships (defined in the mapping, but not used). Once we rebuilt the indexes without PC, average response times returned to the same interval like in 2.4, but still there are high peaks, which weren't in 2.4.

3

What queries?
What is your mapping?

4

Our index mapping is around 2000 lines, so not possible post here. Can I send the mapping and query DSL to your email?

And is it possible that the worse perf is due to migrating old 2.4 indices to 5.2? Can re-build the indices from scratch help resolve the issue?

Also, is there any ES 5.2 specific perf tuning strategy?

5

Use gist/pastebin/etc :slight_smile:

6

Here you go:
https://gist.github.com/youxu71/f8abceb3587be04474ca0d0921195564

Note:

  1. Since the index was created by restoring the old ES 2.4 snapshot, you can see the "string" instead of "text" in mapping.
  2. For some fields like Title, Description, we have multiple language specific fields to better support multi-lingual query, like Title_english, Title_german, etc.
  3. We have nested fields to be used in nested function_score with groovy script_score:
{
"nested": {
	"path": "QueryClicks",
	"query": {
		"function_score": {
			"boost_mode": "replace",
			"query": {
				"match": {
					"QueryClicks.Term.lowercaseraw": "elasticsearch 5.2"
				}
			},
			"functions": [{
					"script_score": {
						"script": {
							"inline": "log(doc['QueryClicks.Count'].value*4)",
							"lang": "groovy"
						}
					}
				}
			]
		}
	}
}
}

More details, please refer to gist.

7

Does anyone encounter similar obvious query performance degradation after migrated to ES 5.2?

8

Could you also share the output of the profile API on that particular query? Ideally both with 2.4 and 5.2 so that we can compare.

9

For the record, there are sme known regressions between 1.x and 2.x but it is the first time I hear of a regression between 2.x and 5.x.

10

After investigation, the degradation might not be due to 5.2 migration. I am still investigating the root cause, and will get back to you with details.

Thanks your help!

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.