Logstash 5.1 - Multiline codec with file input


(Pravin Kumar) #1

Hi All,

I am using Logstash 5.1 , where facing issues with multiline, Since multiline filter is depricated, trying to use multiline codec along with file input.

In the response all the line are getting combined and facing error as in attached file,

PFB the config file and pattern, also sample log file for which used for testing. Also attached the kibana view and logstash log.

FYI, Same is working with multiline filter in earlier version. Logstash_ log is showing few warning as in attachment and throwing grokparsefaiure and multiline tag.

Kindly help to resolve this.

Pattern:***

Logstash pattent configuration

BWTIME %{YEAR} %{MONTH} %{MONTHDAY} %{TIME} GMT +%{INT}
BWLOG %{BWTIME:timestamp} GMT +%{INT} %{GREEDYDATA:Bwpayload}

Config File**

input {
file {
path => "E:/logs/*.log"
type => "esbbwlog"
codec => multiline {
patterns_dir => "E:/Software/ELK/logstash-5.1.1/patterns/ingdevbw"
pattern => "%^{BWTIME}"
negate => true
what => "previous"
}
}
}

filter {
grok {
patterns_dir => "E:/Software/ELK/logstash-5.1.1/patterns/ingdevbw"
match => { "message" => "%{BWLOG}" }
}
date {
match => [ "timestamp" , "YYYY MMM DD HH:mm:ss:SSS" ]
remove_field => [ "timestamp" ]
}
}
output {
if [type] == "esbbwlog" {
elasticsearch {


hosts => [ "IP:9200" ]
index => [ "logstash-bw-devlog-%{+YYYY.MM.dd}" ]
}
stdout { codec => rubydebug }
}
}


(Magnus B├Ąck) #2
pattern => "%^{BWTIME}"

You probably mean:

pattern => "^%{BWTIME}"

(Pravin Kumar) #3

Yeah Magnus.

Sorry i meant pattern => "^%{BWTIME}".

Got the issue resolved. Thanks.
Issue is because of below pattern as GMT +%{INT} is repeated twice.

BWTIME %{YEAR} %{MONTH} %{MONTHDAY} %{TIME} GMT +%{INT}
BWLOG %{BWTIME:timestamp} GMT +%{INT} %{GREEDYDATA:Bwpayload}


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.