How can I expose new fields embodied in top-level fields?

Elastic Stack Kibana
1

Hi all,

I'm using a structured logging package for Go (https://github.com/sirupsen/logrus) and using Filebeat to ship to our ES cluster. Unfortunately, all the structured fields we're exposing with logrus get wrapped up in a "msg" field that I believe Filebeat creates. This makes us unable to query on the structured fields.

For example, in logrus we have:
log.WithFields(log.Fields{
"event": event,
"topic": topic,
"key": key,
}).Fatal("Failed to send event")

But on Kibana, we're reading in log records as such that have a field called 'message' which wraps up the logrus payload:
message: {"event":"click","topic":"video", "key"="key", "level"="fatal", "msg":"Failed to send event","time":"2015-08-12T18:47:07Z"}

Is it possible to extract the event, topic, key fields to top-level?

2

Hi Kpam,

I'm assuming this question is a duplicate of How can I expose new fields sourced from the log record??

Thanks,
CJ