Yes.I am doing a stress test by indexing a lot of documents as soon as possible . My cluster has only on node.
here is the node segment count graph:
here is hot_treads info,( i'm sorry i cann't upload the entire log file). as you can see, there has few merge threads within 19:00 - 19:15 and 19:30 - 19:43 . These two period of time contribute the most disk reads:
[root@localhost es_monitor]# grep -E '2020-04-23-19|cpu usage by thread.*Lucene Merge Thread' hot_threads.log|grep -i -B1 merge|more
2020-04-23-19-07-36 - ::: {localhost.localdomain}{DYjsg8E8RkGXh-ceEPlFBw}{iM51ywJBTxC5ZK6h-VZ5VA}{10.45.154.78}{10.45.154.78:9300}{cname=localhost.localdomain}
48.3% (483ms out of 1s) cpu usage by thread 'elasticsearch[localhost.localdomain][[.monitoring-es-2-2020.04.23][0]: Lucene Merge Thread #60]'
2020-04-23-19-50-34 - ::: {localhost.localdomain}{DYjsg8E8RkGXh-ceEPlFBw}{iM51ywJBTxC5ZK6h-VZ5VA}{10.45.154.78}{10.45.154.78:9300}{cname=localhost.localdomain}
1.9% (19.3ms out of 1s) cpu usage by thread 'elasticsearch[localhost.localdomain][[testshardread][1]: Lucene Merge Thread #0]'
2020-04-23-19-50-44 - ::: {localhost.localdomain}{DYjsg8E8RkGXh-ceEPlFBw}{iM51ywJBTxC5ZK6h-VZ5VA}{10.45.154.78}{10.45.154.78:9300}{cname=localhost.localdomain}
10.1% (100.7ms out of 1s) cpu usage by thread 'elasticsearch[localhost.localdomain][[testshardread][1]: Lucene Merge Thread #0]'
2020-04-23-19-50-48 - ::: {localhost.localdomain}{DYjsg8E8RkGXh-ceEPlFBw}{iM51ywJBTxC5ZK6h-VZ5VA}{10.45.154.78}{10.45.154.78:9300}{cname=localhost.localdomain}
10.6% (106ms out of 1s) cpu usage by thread 'elasticsearch[localhost.localdomain][[testshardread][1]: Lucene Merge Thread #0]'
2020-04-23-19-50-50 - ::: {localhost.localdomain}{DYjsg8E8RkGXh-ceEPlFBw}{iM51ywJBTxC5ZK6h-VZ5VA}{10.45.154.78}{10.45.154.78:9300}{cname=localhost.localdomain}
8.5% (85.2ms out of 1s) cpu usage by thread 'elasticsearch[localhost.localdomain][[testshardread][1]: Lucene Merge Thread #0]'
2020-04-23-19-50-53 - ::: {localhost.localdomain}{DYjsg8E8RkGXh-ceEPlFBw}{iM51ywJBTxC5ZK6h-VZ5VA}{10.45.154.78}{10.45.154.78:9300}{cname=localhost.localdomain}
9.2% (91.9ms out of 1s) cpu usage by thread 'elasticsearch[localhost.localdomain][[testshardread][1]: Lucene Merge Thread #0]'
2020-04-23-19-50-57 - ::: {localhost.localdomain}{DYjsg8E8RkGXh-ceEPlFBw}{iM51ywJBTxC5ZK6h-VZ5VA}{10.45.154.78}{10.45.154.78:9300}{cname=localhost.localdomain}
9.4% (94.4ms out of 1s) cpu usage by thread 'elasticsearch[localhost.localdomain][[testshardread][1]: Lucene Merge Thread #0]'
2020-04-23-19-51-00 - ::: {localhost.localdomain}{DYjsg8E8RkGXh-ceEPlFBw}{iM51ywJBTxC5ZK6h-VZ5VA}{10.45.154.78}{10.45.154.78:9300}{cname=localhost.localdomain}
8.7% (86.8ms out of 1s) cpu usage by thread 'elasticsearch[localhost.localdomain][[testshardread][1]: Lucene Merge Thread #0]'
2020-04-23-19-51-02 - ::: {localhost.localdomain}{DYjsg8E8RkGXh-ceEPlFBw}{iM51ywJBTxC5ZK6h-VZ5VA}{10.45.154.78}{10.45.154.78:9300}{cname=localhost.localdomain}
3.7% (37.2ms out of 1s) cpu usage by thread 'elasticsearch[localhost.localdomain][[testshardread][1]: Lucene Merge Thread #0]'
2020-04-23-19-51-04 - ::: {localhost.localdomain}{DYjsg8E8RkGXh-ceEPlFBw}{iM51ywJBTxC5ZK6h-VZ5VA}{10.45.154.78}{10.45.154.78:9300}{cname=localhost.localdomain}
4.6% (45.5ms out of 1s) cpu usage by thread 'elasticsearch[localhost.localdomain][[testshardread][1]: Lucene Merge Thread #0]'
2020-04-23-19-51-07 - ::: {localhost.localdomain}{DYjsg8E8RkGXh-ceEPlFBw}{iM51ywJBTxC5ZK6h-VZ5VA}{10.45.154.78}{10.45.154.78:9300}{cname=localhost.localdomain}
1.2% (11.9ms out of 1s) cpu usage by thread 'elasticsearch[localhost.localdomain][[testshardread][1]: Lucene Merge Thread #0]'
I'd love to believe that too much disk reads because of merge operation,but i can not find evidence to prove it.
