4 Server, each with 2 cpus (each 24 vCore) 256 GB Ram, 2 SSDs (each 120GB) Raid0, 10Gbit interconnect.
On each Server we have 4 ES 6.1.1 Docker-Container running, created with docker compose. Each ES-Node has 29GB Heap. Docker has no limits to Hw.
Now we wanted to reindex an index with 120GB of Data (16 shards, 1 replica). No other tasks are running on the servers or nodes. The cluster can use the complete Hardware exclusively for reindexing.
After we started the reindexing task, we were wondering why the cluster is not realy using the hardware. Only some vcores on the servers were running with around 10%, the SSD I/O were between 0 and 20% and the Lan interconnect has been used minimally. Finally it took 3 hours for the task.
Can sombody tell us, why the hardware is not really used while reindexing?