Help parsing nested JSON with dynamic top key

Elastic Stack Logstash
1

Hi everyone,

I am looking for help to tackle a fairly difficult problem for an unexperienced logstash user.

I have an application that pushes JSON messages to redis.

The message format is the following:
http://pastebin.com/gRyRAejb

I need to insert the contents of each message in ElasticSearch, but I want to get rid of the top level key (in the example called 'DYNAMIC_TOP_KEY', so I only want the values inside.

I don't really know how to perform this operation in logstash filters. I also cannot access the fields below as I don't know how to capture the top level key.

For example if I want to delete the "msg_name", how should I do that?

Thanks a lot for the help in advance!

2

You'll have to use the ruby filter to copy key/value pairs of nested fields into the top level. This appears to work:

filter {
  ruby {
    code => "
      event.to_hash.clone.each_value{|v|
        if v.is_a? Hash
          v.each_pair{|k,v|
            event[k] = v
          }
        end
      }
    "
  }
}