Hi Team
I have a self managed elastic cluster. We are using Filebeat -> logstash -> Elasticsearch cluster-> Kibana setup. All 7.15v with basic license.
Data transferred through out the day is around 260GB. Filebeat have four input type : log for 4 different file and is configured as service in Centos box. Using load balance : true it is shipping logs to two logstash instances. Using scan_frequency of 1sec and bulk_max_size is 1000.
At peak time we are seeing delays in some log file inputs, i.e. out of 4 one is logged properly for others there are delay, being added in elastic and viewable in kibana.
Please suggest.