Nested JSON parsing in dynamic field names

Elastic Stack Logstash
2

Do you want to split the event into multiple events, each of which has a username, age, and occupation?

If so, you'll need to do a sequence of things, using the [@metadata] namespace (which is not exported from Logstash by default):

  • Use the JSON Filter to extract the object from the JSON string into [@metadata][string-indexed]
  • Convert the string-indexed object into an Array ([@metadata][items-array])
  • Split the Event on the resulting array, putting the result at [@metadata][single-item]
  • Do some cleanup to move the relevant sub-keys up to the top namespace.
filter {
  json {
    source => "message"
    target => "[@metadata][string-indexed]"
  }
  ruby {
    code => "
      # extract the string-indexed object to a local variable
      string_indexed = event.get('[@metadata][string-indexed]')
      # iterate over the keys in the order of the string-encoded integer index
      items_array = string_indexed.keys.sort_by(&:to_i).map do |key|
        # get the item from the indexed map
        string_indexed[key]
      end
      # set the items array.
      event.set('[@metadata][items-array]', items_array)
    "
  }
  split {
    field => "[@metadata][items-array]"
    target => "[@metadata][single-item]"
  }
  mutate {
    rename => {
      "[@metadata][single-item][username]" => "username"
      "[@metadata][single-item][age]" => "age"
      "[@metadata][single-item][occupation]" => "occupation"
    }
  }
}