Indexing performance of V2.3.5

Elastic Stack Elasticsearch
1

Hi all, we are evaluating indexing performance of V2.3.5, comparing with V1.7.5, the cluster nodes of the two versions are the same as:

o 1 client node
o 8 data nodes
o 3 masters nodes
    o standard analyzer
    o the same source documents on the same index
    o same mappings and settings including shard number and replicas

we are using scroll API on a jump box node inside the cluster to do indexing test and getting below data:

in V1.7.5 indexing time of tests by locales:

en-us,02:16:36

ar-eg, 01:49:51

ru-ru, 01:00:08

en-gb, 00:54:31

tr-tr, 00:45:51

in V2.3.5 indexing time of tests by locales:

en-us, 03:13:30

ar-eg, 02:37:54

ru-ru, 01:27:39

en-gb, 01:15:35

tr-tr, 01:05:26

as can be seen indexing time of all tests grows in V2.3.5 test.

I saw there's change about doc value default change write on disk in V2.3.5 which will bring up to 25% slower performance:
https://www.elastic.co/guide/en/elasticsearch/guide/1.x/doc-values.html
https://www.elastic.co/guide/en/elasticsearch/reference/current/doc-values.html

but above performance data in our testing shows it's much more than 25%. (like en-us one it's 50%), any one can help explain why this happened in our testing?

or is there any default changes in V2.X we need to care in this comparison?

Thanks!
Tony

2

In the interests of improved durability there was a change to the settings around fsync frequency. If you want to optimise for performance you can relax these settings but be aware of the implications - see https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-translog.html#_translog_settings