I am using a csv processor and \t as the separator ingest pipeline to push logs from filebeat. This works great but fails when the message (logmessage) portion has a new line character. I added the following to the filebeat.yml to deal with multiline but hasn't helped. Loglines start with a TS like 2020-12-29T08:25:01.971....
Any help would be great?
filebeat.yml
multiline.type: pattern
multiline.pattern: '^20'
multiline.match: after
multiline.negate: true
pipeline def
Copy to clipboard
"pipeline_tab" : {
"description" : "tab pattern",
"processors" : [
{
"csv" : {
"field" : "message",
"target_fields" : [
"timestamp",
"relativeTime",
"thread",
"processName",
"sourceName",
"logType",
"logMessage"
],
"separator" : "\t"
}
}
]
}