Elasticsearch not indexing while rebalancing

Elastic Stack Elasticsearch
1

Hello,

I have a 6 node cluster on AWS split with allocation awareness to 2 racks.
I wanted to replace the nodes on 1 rack to increase disk size, so I launched 3 new nodes on that rack and excluded allocation for the departing nodes.

Everything seems to be in order, cluster is GREEN, shards are reallocating, but according to Marvel index rate is 0/s, and logstash is also reporting the pipeline is blocked but no errors from ES.

Any idea what may be causing this?

Omer

2

What version are you running?
Have you made changes to recovery/reallocation?

3

right, sorry

Using 2.3.3
cluster.routing.allocation.node_initial_primaries_recoveries: 5 cluster.routing.allocation.node_concurrent_recoveries: 15 cluster.routing.allocation.cluster_concurrent_rebalance: 20 cluster.routing.allocation.awareness.attributes: rack_id

4

I'm experiencing an identical problem right now...

version 2.2
node_concurrent_recoveries: 30
cluster_concurrent_rebalance: 30
everything else defaults

5

Maybe it's because the write consistency level:
https://www.elastic.co/guide/en/elasticsearch/reference/2.4/docs-index_.html#index-consistency