So the software we are using generate log files in different format and throw everthing in one big log file, so I am dealing with this one big log file with different types of logs, they all share the first couple fields(timestamp. hostport etc.). I know we have the if/else expression as well as the type settings in file plugins. But how do I make them work together, are there ways to assign types to different types of logs in one log file in the input section? or there are other options.
Thank you so much!
type of just another field, so you can change it after peeking at the contents of the line. Or you can use tags. Or just big conditionals. If the different log lines will have different fields I'd go with different type values, probably assigned via conditionals.
Thank you so much for helping out, btw the last post you helped me with solved my problem, I really appreciated.
How would I know what field I have and assign types in the input section? I thought you only get to know what fields you have after you parse them in filter section?
How would I know what field I have and assign types in the input section?
You don't, and it doesn't matter. Use conditionals and filters for picking a suitable type value based on what each message looks like.