I am having several servers (windows) running filebeat to get log files and send events to the Logstash.
Logstash and Elasticsearch runs on the same Linux machine.
I have developed a monitor scheduled script on the windows servers, which every 5 minutes writes a line into a monitored file which filebeat reads it and sends data to the Logstash. The Logstash takes the data and insert into a specific index.
sometimes that heartbeat data is being analyzed by Logstash, after 2 or 3 minutes it was written to the monitored log file, which can be caused by an overloaded Logstash process.
- How can I monitor the Performance of the Logstash and Indexing in Elasticsearch ?
- what are best practices for cases when number of Input plugins is huge.
5 Servers of type I
16 Servers of type II
10 Servers of type III
3 Servers of type IV
(*) Servers of type I, has many prospectors (15) in the Filebeat config.
Would like to have best practices for such a config.