I'm continuing to investigate this. A few things I've discovered:
-
Setting the refresh_interval to -1 to disable it, setting disable_flush to true, either independently or together does not seem to impact the frequency of merge throttling
-
Various changes to merge_floor do not seem to significantly impact the frequency of merge throttling
-
Increasing, or decreasing the number of shards may somewhat reduce or increase the frequency of merge throttling, but it seems like the duration or contracts relative to the duration of the frequency
Despite tweaking these settings, merging seems to cause throttling around every 5 minutes, almost on the dot. This gives an example of frequency of 'now throttling' over the last hour:
16:01 10
16:08 33
16:15 11
16:17 71
16:21 27
16:31 63
16:32 20
16:37 57
16:38 26
16:43 55
16:44 28
16:49 16
16:50 20
This configuration is barely touching the disk. and has a fairly low indexing rate... Why is it merging so much?