Hello,
I tested a certain pattern in kibana debugger:
As you can see, it works fine, but when sending the same log to logstash I always end up getting grok parse failure, how can I check what the issue is...?
This is my logstash configuration file:
input {
udp {
port => 5960
codec => plain {
charset => "UTF-8"
}
type => "log4net"
}
}
filter {
if [type] == "log4net" {
grok {
break_on_match => true
remove_field => message
match => {
message => "(?m)\[%{GREEDYDATA:datetime}\] \[%{LOGLEVEL:level}\] \[%{DATA:component}\] \[%{NUMBER:thread}\] \[%{GREEDYDATA:parameters}\] %{GREEDYDATA:tempMessage}((\r\n)|(\n))(?<exceptionType>(((%{JAVACLASS})|(System.))Exception)): (?<exceptionMessage>(%{GREEDYDATA}))((\r\n)|(\n))(?<stackTrace>(( )+at %{GREEDYDATA}))"
}
match => {
message => "(?m)\[%{GREEDYDATA:datetime}\] \[%{LOGLEVEL:level}\] \[%{DATA:component}\] \[%{NUMBER:thread}\] \[%{GREEDYDATA:parameters}\] %{GREEDYDATA:tempMessage}((\r\n)|(\n))(?<exceptionType>(((%{JAVACLASS})|(System.))Exception)): (?<exceptionMessage>(%{GREEDYDATA}))"
}
match => {
message => "(?m)\[%{GREEDYDATA:datetime}\] \[%{LOGLEVEL:level}\] \[%{DATA:component}\] \[%{NUMBER:thread}\] \[%{GREEDYDATA:parameters}\] %{GREEDYDATA:tempMessage}"
}
}
if !("_grokparsefailure" in [tags]) {
mutate {
replace => [ "message" , "%{tempMessage}" ]
}
}
mutate {
remove_field => [ "tempMessage" ]
}
}
}
output {
elasticsearch {
hosts => ["localhost"]
manage_template => false
index => "ovc-%{+YYYY.MM.dd}"
}
}
Example input:
[22/02/2020 13:32:32.6916] [INFO] [L2OvC_WebAPI.Loggers.Interfaces.ICachedAttributeLogger] [29] [Controller= Action=] No cached value found, proceeding with request.
Kibana output:
Is it maybe because message looks weird when viewing the json format? I tried to paste that into kibana debugger and it failed:
