I have a syslog and ELK stack setup that I am working though the final design of. A brief overview of what I am doing, I have about 100 firewalls from 10+ different vendors that I am sending data to a syslog server. The syslog server stores the raw data and copies the messages to my separate ELK stack server. I am going to need to do quite a bit of massaging of the data I receive to make the data uniform and meaningful. Due to my rules of engagement I have some limitations in how I can configure the firewalls so using alternate ports for syslog messages is not something I can do.
With this in mind I am quickly realizing that my config file will get pretty large very quickly. I know that separate config files in logstash are concatenated into one file. I would like to have separate files for each firewall vendor to make for easier editing of the files when needed. So my question is two part, how does logstash decide the order to concatenate the files? Is logstash okay with a very large config file of say a 1000 lines, or should I devise a way to run multiple instance of logstash?
Thanks in advance.