Hi,
since i've moved multiline management from filter layer to input layer, i've doubts on how to do it correctly using lumberjack or the new version of beats (that ive never used before)
In production enviroment actually we pick logs from forwarders on many hosts. For now the pipeline has 1 lumberjack input, that collects from all the forwarders. Each forwarder ships many log type with this configuration:
"files": [
{
"paths": [
"/var/log/httpd/access_log"
],
"fields": {
"type": "ssl_access_log",
"host": "camel_1"
}
},
{
"paths": [
"/var/log/httpd/camel.log"
],
"fields": {
"type": "camel_log",
"host": "camel_1"
}
},
{....
Each event is tagged with is own type, so i can discriminate later and apply the proper multiline transformation (in filter layer)
How would the multiline management become with lumberjack / beats input?
Which best practice should i apply to avoid different-log-type mixing and to ensure event uniqueness?
thank so much