We are setting up elasticsearch, kibana, logstash and filebeats on a server to analyse log files from many applications. Due to reasons* each application log file ens up in a separate directory on the server. We have about 20.
As I understand we can run a logstash pipeline config file for each application log file. That will be one logstash instance running with 20 pipelines in parallel and each pipeline will need its own port beat port. Please confirm that this is correct?
Can we have one filebeat instance running or do we need one for each pipeline/logfile?
Is this architecture ok or do you see any major down sides?
- There are different vendors responsible for different applications and they run a cross many different OS and many of them will not or can't install anything like filebeats.