I am trying to figure out how to deal with different types of log files using Filebeats as the the forwarder.
Basically I have several different log files I want to monitor, and then I actually want to put an extra field in to identify which log the entry came from, as well as a few other little things. This is then forwarded onto Logstash for further processing, which is where each element comes into play.
My problem is that it doesn't seem to play nicely once you add more than one file. Usually the last entry is the one that is uses. The documentation is confusing as well, in regards to how to achieve it, with document_type and input_type being interchanged.
List of prospectors to fetch data.
# Each - is a prospector. Below are the prospector specific configurations
- paths: - "/www/sites/logs/dog.log" document_type: log fields: type: dog generator: doglog server: myserver - paths: - "/www/sites/logs/cat.log" document_type: log fields: type: cat generator: catlog server: myserver