Performance hit when multiple filebeats are sending to same ES

Elastic Stack Beats
22

Hmm, this is even weirder, I would've expected to see higher throughput rates at this point.

A few more ideas:

  1. What happens if you disable replicas on the index?
    • Note: This isn't something you should do for production, but I'm curious if it alters your indexing rate.
  2. In your filebeat.yml when happens if you use the following settings:
queue.mem.events: 65536
queue.mem.flush.min_events: 2048

output.elasticsearch.bulk_max_size: 500
output.elasticsearch.worker: 16

Note: You might want to mess around with the output.elasticsearch.bulk_max_size and output.elasticsearch.worker settings. Every cluster is different when it comes to the ideal values for bulk_max_size and workers.

  1. What happens now if you have 2 or 3 Filebeats pointed at your 3 node cluster?