I have been running different capacity and performance test scenarios on our ELK cluster and today I have noticed something that didn't make sense to me.
I had 4 data nodes and 2 coordinating nodes that I send my LS output to. Each index has 4 primary shards and 1 replica.
I was able to see a constant primary indexing rate of 3k /s coming from 3 different LS nodes.
The data is metricbeat data nothing beyond that.
I wanted to be able to increase my indexing rate so I have decided to add two more data nodes so now I have 6 and increased number of primary shards to 6.
Any idea why once the 2 extra data nodes were added, with the same exact settings as the other 4, my indexing rate has dropped from 3k to around 800 /s ?