Multiline Pattern not Working

Hi Team ,

I am trying to configure filebeat to ship logs to logstash from one server.

Filebeat is harvesting all the lines together irrespective of MultilineConfiguration.

Sample Logs:

2019-09-09T23:40:59,919 | INFO | 22 - timer://foo | Sample23 | ID-co-in-1567014689815-80-423521212 | SALESMAN_CODE=null}
2019-09-09T23:41:48,922 | INFO | 22 - timer://foo | Sample23 | ID-co-in-1567014689815-80-41212122 | SALESMAN_CODE=100}

Filebeat Config :

Multiline options

Multiline can be used for log messages spanning multiple lines. This is common

for Java Stack Traces or C-Line Continuation

The regexp Pattern that has to be matched. The example pattern matches all lines starting with [

multiline.pattern: ^[0-9]{4}-[0-9]{2}-[0-9][A-z]{5}

Defines if the pattern set under pattern should be negated or not. Default is false.

multiline.negate: true

Match can be set to "after" or "before". It is used to define if lines should be append to a pattern

that was (not) matched before or after or as long as a pattern is not matched based on negate.

Note: After is the equivalent to previous and before is the equivalent to to next in Logstash

multiline.match: after

Logstash Config :

input {
beats {
port => "5044"
}
}
filter {

    grok {
            match => [ "message" , "%{TIMESTAMP_ISO8601:timestamp}%{SPACE}\|%{SPACE}%{WORD:LogLevel}%{SPACE}\|%{SPACE}%{GREEDYDATA:Word} \|%{SPACE}%{WORD:Routename}%{SPACE}\|%{SPACE}%{GREEDYDATA:ID}%{SPACE}\|(?m)%{GREEDYDATA:payload}" ]
}
      
date {
    match => [ "timestamp", "ISO8601","YYYY MMM dd HH:mm:ss:SSS" ]
    remove_field => [ "timestamp" ]
    }

}
output {
elasticsearch {
hosts => [ "192.168.43.116:9200" ]
index => "logstash-sample-esb-%{+YYYY.MM.dd}"
user => elastic
password => password
}

}

Kindly help!!!

Regards,
Mugil

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.