Performance issue with replication enabled (5.1.1)

In a spinning drive set up, we've seen the performance go down 8X after replication is enabled. This is too much a hit for us. I've read a published performance tip that recommends turning off replication then then it back on to avoid duplication indexing. But we would really love to be able to continuously replicate our data.

Our set up is two instances on spinning drives. The IO wait is maxed and appears to be the bottleneck. With duplicate indexing we'd expect to close to 2X penalty but any explaination why it's 8X?

How are you measuring this performance?

To measure the system throughput, we are driving a known set of events through filebeat collectors, and then observing the elasticsearch performance. When the system is "healthy" the elasticsearch will be able to index the event traffic as it comes in. When the performance limit is reached, the indexing falls behind the event traffic, and events back up in the input queue.

With replication disabled (replication=0), we are ale to drive 8x the number of events vs when we have replication set to 1.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.