My current filebeat configuration reads logs from two different directories. Each directory contains logs of distinct formats, therefore I have two log formats.
For each directory, I have specified a filebeat prospector and I have assigned distinct tags.
Apparently, I cannot assign distinct shipping destinations (note: I am mostly interested in distinct ports) per prospector. This means that Logstash will retrieve all logs via the same port and will attempt to apply a single multiline pattern for the initial parsing of logs.
Unfortunately, this will not work in my case since two patterns are required (one for each log format). What makes it worse is that the pattern for one of the log formats is rather unknown/variable.
Any suggestions? Should I have to go with two filebeat installations?