Indices.store.throttle.type: none and max_bytes_per_sec (ES 1.7.x), should I raise max_bytes_per_sec on RAID?

Elastic Stack Elasticsearch
1

I'm having issues with merge throttling indexing.

Setting "indices.store.throttle.type" to "none" doesn't seem to prevent throttling if max_merge_count is exceeded. I don't have SSDs, but the server that I am using has RAID which has a 1200MB/s throughput.

If I set throttle type to none, does max_bytes_per_sec still impact the merge rate?

If I tweak the max_bytes_per_sec value, it shows it defaulting to a very low 20MB/s.

If so, might I see a reduction in index throttling if I raise the max_bytes_per_sec to something higher?

2

Closest reference I've found so far ... Excessive merging/small segment sizes

3

I've (tried) to read through some of the ES code (StoreRateLimiting,RateLimitedFSDirectory) and into some Lucene code (SimpleRateLimiter) and my interpretation is that the rate limitation is not in effect if the merge type is set to none. Does this sound right?

4

unknownunknown,

That is correct. with indices.store.throttle.type set to none, any value for indices.store.throttle.max_bytes_per_sec is not used.

Glen

1 Like
5

Thanks, this confirmation is helpful!