Multiline FIlebeat configuration for custom log

Hi , I have successfully set up real time centralized logging. The only issue that remains is the custom logs. It has mutiple lines for a single event. Since my shipper(filebeat) sends these logs to the buffer(Redis), I need to handle these multi line events in the filebeat end and not in the logstash end. Here is how the log lines look

1/11/2017 12:00:17 AM :
Error thrown is:
No Error


Request sent is:
webMethod:GetOSSUpdatedOrderHeader|appCode:OSS|regionCode:EMEA|orderKeyList:|lastModifedDateTime:1/10/2017 11:59:13 PM|


1/11/2017 12:00:23 AM :
Error thrown is:
No Error


Request sent is:
webMethod:GetOSSUpdatedOrderHeader|appCode:OSS|regionCode:APJ|orderKeyList:|lastModifedDateTime:1/10/2017 11:58:17 PM|


and here is the filebeat.yml config:

exclude_lines: ["^*"]
multiline.pattern: "(\d{1,2})/(\d{1,2})/(\d{4})(\d{2}):(\d{2}):(\d{2})\s(AM|PM]\s:"
multiline.negate: true
multiline.match: after

I have tried several other patterns to let filebeat know that the date field is where I want the new event to start from. However, I get the entire event being made into a single line all the time.'Please help me out here

Using single quotes when writing regular expressions in your config file. Take a look at the tester where I modified your expression: https://play.golang.org/p/Mw9Mq1Some

https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html

@andrewkroh Thanks a ton, I had been breaking head on this one for quite a while!!!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.