Hi All, when harvesting logs from my machines to logstash using filebeat I noticed logs are not delivered fast enough, especially when using TRACE logs and adding more machine (filebeats).
My stack is something like ~40 machines each writing ~20 logs per second and one logstash with 2 CPU and 4GB RAM (aws t2.medium) - machine metrics looks good and it's not looks like logstash having tough life while processing the logs, but still I can see delay of something like ~1 hour of logs being harvested and send to logstash.
What is the ratio of logs per sec to number of logstash ? how much logstash instances needed? what size of logstash instances ? how can I make filebeat harvest faster ?