How to dynamically move nested key value to root level

This is my logstash config file:

input {
   http {
    id => bulkHttpInput
    port => 8088
    additional_codecs => {"application/json" => "es_bulk"}
    codec => es_bulk
  }
}
filter {
  mutate {
    remove_field => [ "headers"] 
  }
}
output {
  elasticsearch {
   id => elasticOutputOfHttp
   index => "%{[@metadata][_index]}"
   document_type => "%{[@metadata][_type]}"
   document_id => "%{[doc][docID]}"
   doc_as_upsert => "true"
  }
}

and the output is like :
{
"host" => "127.0.0.1",
"@timestamp" => 2019-05-07T14:22:58.364Z,
"@version" => "1",
"doc" => {
"field1" => "my_field1",
"field2" => "my_field2",
"field3" => "my_field3"
}
}

I wanna move all fields within doc to root level. This is possible by defining add_field and move all nesteds fields from doc but field names are dynamic for-example "field4" may be added and some other fields may be removed . How to do it dynamically?

    ruby {
        code => '
            event.get("doc").each { |k, v|
                event.set(k,v)
            }
            event.remove("doc")
        '
    }
10 Likes
How to add/rename fields recursively instead using mutate or instead doing one by one?
How To parse Json in json
Logstash filter to remove a parent in a big array
Splitting HTTP_Poller JSON Results
Parsing dictionary into json file
Parsing Nested JSON with Array
Generating events from JSON array inside root-level object
Json input and splitting
Rename nested field name
How to remove the root element while xml parsing using Logstash?
Move all fields to a subfield?
Logstash Json filter re-formated the @timestamp field unexpectedly
One doc for each array item
Parsing multiple lines as multiple events using fields from first and last line
Event parsing - perf_data from cmkbeat
Logstash Filter - Breaking up fields into multiple documents
Parsing json object
How to extract data from multiline(codec)
Logstash JSON parse error
Logstash xml input configuration: for multiple documents
Add two value from two record
Logstash move subfield to top field
How changing root of json events with logstash filter?
Problem with data merging
Parsing csv file through Logstash
Split objects in a nested array into separate record
Getting sum of a field in xml
Working with key:value arrays
Processing Multiple Rows From JDBC Streaming
Ingest xml file using Logstash
Parsing issues
How to rename some fields by using regex statement
Logstash parse object
JSON Parsing Issue
Routing one input to diferent indexes
Logstash extract a nested json object
Dynamic JSON parsing
Parse and extract json fields using logstash
Parsing nested json to flat structure

It works,
Thank you

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.