Logstash - Add fields from the log fields property - Grok

Elastic Stack Logstash
1

I'm learning logstash and I'm using Kibana to see the logs. I would like to know if is there anyway to add fields using data from message property.

For example, the log is like this:

port:46,722 host:172.18.0.5 message:{"@timestamp":"2016-12-28T00:34:53.198+00:00","@version":1,"message":"Entrada de datos incompletos","logger_name":"com.empresa.miAlquiler.controllers.UserController","thread_name":"http-nio-7777-exec-2","level":"INFO","level_value":20000,"HOSTNAME":"8fe48aff9ca8","spring.application.name":"visit-service","X-Span-Export":"false","X-B3-SpanId":"1cccc5c7252100c4","X-B3-TraceId":"1cccc5c7252100c4"} tags:_grokparsefailure

message, is a field. I want to get spring.application.name from message and convert it as a field, to filter the logs with it with Kibana.

How can I do that?

My Logstash conf file, looks like:

filter {
grok {
match => {
"message" =>
"^%{TIMESTAMP_ISO8601:timestamp}\s+%{LOGLEVEL:level}\s+%{NUMBER:pid}\s+---\s+[\s*%{USERNAME:thread}\s*]\s+%{JAVAFILE:logger_name}\s*:\s*%{DATA:themessage}(?:\n+(?(?:.|\r|\n)+))?$"
}
}
date {
match => [ "timestamp" , "yyyy-MM-dd HH:mm:ss.SSS" ]
}
mutate {
remove_field => ["@version"]

add_field => {"service_name" => "%{spring.application.name}"}

}
}

Thanks! :slight_smile:

2

Do not parse JSON strings with grok filters. Either change your input's codec to json or json_lines or use a json filter to parse the message field.

1 Like
3

It works! Thank you very much! :slight_smile:

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.