Convert object/array datastructures to nested

[jonasbareiss]

Hi everyone,
I have an elasticsearch index that was used as datastore, where only the source fields were used. Some data is in an object structure, but now I need to be able to aggregate on it. The array does not always have the same number of elements.

Is there any method to "convert" or copy over the fields, maybe with mutate.
Or is the only possibility a Ruby script to cycle through the array

Here is the Rubydebug output of a data sample:

    "attributes" => {
        "articles" => {
                "code" => "articles",
            "children" => {
                "techData" => [
                    [0] {
                        "__identity" => "a25337ab-be66-dd0f-5fd2-5aa6f338697f",
                              "code" => "MATNR",
                           "sorting" => "",
                            "values" => {
                            "data" => {
                                "text" => "9999999"
                            }
                        },
                              "name" => "Artikelnummer",
                             "count" => "1",
                            "format" => "techData"
                    },
                    [1] {
                        "__identity" => "0607beaa-1bf9-4020-2f47-d2850c3546ae",
                              "code" => "PL_DYN_QUERSCHNITT",
                           "sorting" => "2",
                            "values" => {
                            "data" => {
                                "text" => "2,5"
                            }
                        },
                              "name" => "Leiterquerschnitt in mm²",
                             "count" => "1",
                            "format" => "techData"
                    },
                    [2] {
                        "__identity" => "dfd12f5a-77b6-e1c7-8655-12131883fe97",
                              "code" => "PL_DYN_DURCHMESSERAUSSENMINMAX",
                           "sorting" => "6",
                            "values" => {
                            "data" => {
                                "text" => "3.4 - 4.1"
                            }
                        },
                              "name" => "Außendurchmesser in mm",
                             "count" => "1",
                            "format" => "techData"
                    },
                    [3] {
                        "__identity" => "2ee54785-08ec-a9c3-35c6-a19903f1d49e",
                              "code" => "PL_COLOUR",
                           "sorting" => "8",
                            "values" => {
                            "data" => {
                                "__identity" => "fe3169e9-abee-44cc-5ffe-e8ca2ba13adb",
                                      "text" => "braun"
                            }
                        },
                              "name" => "Aderfarbe",
                             "count" => "1",
                            "format" => "techData"
                    }
                ]
            }
        }
    }

I would need to get just the "Code" or the "name", and the "[data][text]"

The output should be sth like this:

"techData":{
   "key": "MATNR"
   "value" : "9999999"
}
{
   "key": "PL_COLOUR"
   "value" : "braun"
}

I found this post:

I would like to access nested data from an array

I think the major issue here is that the source application is sending essentially a hash in array form. If the counters element was: "counters": { "n_lost_packets": 0, "n_sent_bytes": 130852808 } instead of: "counters": [ { "name": "n_lost_packets", "value": 0 }, { "name": "n_sent_bytes", "value": 130852808 } ] it would be simpler. If you want to programatically you need ruby filter since, in your case each hash contains a key under "key" and a value under "valu…

but I am not sure how to get all the needed data back