Index_buffer_size vs max_index_buffer_size

Elastic Stack Elasticsearch
1

I'm currently running ES 1.7.1 although the docs for this haven't changed so I'm not sure that's relevant.

The docs for these settings give a fairly decent explanation of these settings but it's a bit unclear why max_index_buffer_size would be required if index_buffer_size has been set.

indices.memory.index_buffer_size
Accepts either a percentage or a byte size value. It defaults to 10%, meaning that 10% of the total heap allocated to a node will be used as the indexing buffer size shared across all shards.

indices.memory.min_index_buffer_size
If the index_buffer_size is specified as a percentage, then this setting can be used to specify an absolute minimum. Defaults to 48mb.

indices.memory.max_index_buffer_size
If the index_buffer_size is specified as a percentage, then this setting can be used to specify an absolute maximum. Defaults to unbounded.

Example:
ES heap size is 10g
index_buffer_size is set to 10%

I'd assume that implies ES will share 1g of heap among the shards currently indexing.
I'd also assume that implies a max of 10% (1g) but then why would max_index_buffer_size exist.

Questions:

  1. Does "shared across all shards" imply just shards from indices currently doing indexing or does this imply all shards from all open indices share this memory?

  2. How does index_buffer_size truly differ from max_index_buffer_size?

2

The min and max are taken into account only when index_buffer_size is a percentage. In that case the min and max may override index_buffer_size depending on how much memory is available.

I'll see if we can get the docs updated to reflect this.