Hi,
I have an Elasticsearch cluster with 24 data nodes. I have about 35 logstash pipelines on a single server that has 4 CPUs and 16 GB RAM. Each Logstash pipeline has 18 filters w/ two of them being Elasticsearch filters for lookups. Is there a minimum hardware spec or best practice for this kind of work load? I've upgraded the EC2 instance to have 8 CPUs and 32GB RAM, but it still seems to utilize 400% of CPU. We're planning to load balance to multiple Logstash servers, but I'm in need of a quick fix for now.