Hello,
I am currently running a Proxmox + Ceph cluster. It is backed by 17 256gb SSDs. I have noticed that my iowait times for CPU were terrible. a 2 core VM was spending 50-75% of it's cpu on IOWait. The only thing running was my base services(monitoring).
Filebeat was included. Once I turned filebeat off on my 50 VMs, performance significantly increased across the entire cluster. Most of my filebeat config settings are standard. Any thoughts?
Also worth noting is my Elasticsearch cluster uses NFS storage for ES storage so it wasn't an issue of ES consuming all the write capacity. Logstash does use persistent queues so maybe that was a large impact?