HTTP Filter - Unnest Array

Hi there,
I need to unnest an array (based on the workaround mentioned in this previous thread) into seperate documents.

I am calling an API and getting in return an array of results. Each of these results should preferably be seperated into its own document.

Currently all these results are stored inside a body field (see workaround, this seems to be the only option atm ). I assume that you could probably loop over each element in the array and then create a new document for each, with a ruby script. Unfortunanely I never wrote anything Ruby related let alone Ruby + Logstash.
Maybe someone could point me to a good starting point and/or give me an example if possible.

Here is the current output as reference:

{
    "@timestamp" => 2021-07-05T06:25:43.321Z,
        "origin" => "XXX",
      "@version" => "1",
       "headers" => {
                             "date" => "Mon, 05 Jul 2021 06:25:44 GMT",
        "strict-transport-security" => "max-age=15724800; includeSubDomains",
                     "content-type" => "application/json",
                    "cache-control" => "max-age=0, private, must-revalidate",
                             "vary" => "Accept-Encoding",
                     "x-request-id" => "24e8d9f20bf78f24f586980a17d47cef",
                       "connection" => "keep-alive",
                "transfer-encoding" => "chunked",
                         "per-page" => "100",
                        "x-runtime" => "0.023585",
                            "total" => "9",
                             "etag" => "W/\"5f5dd0d16cda9863332b6bac2480dffb\""
    },
        "apiurl" => "XXX&date=2021-07-05",
          "body" => [
        [0] {
            "received_at" => "2021-07-05T07:39:47.000+02:00",
                 "length" => 4,
                 "amount" => 0.118,
             "answer_uri" => nil,
                     "to" => "XXX",
            "description" => "XXX ",
                   "from" => "XXX"
        },
        [1] {
            "received_at" => "2021-07-05T07:41:33.000+02:00",
                 "length" => 587,
                 "amount" => 1.18,
             "answer_uri" => nil,
                     "to" => "XXX",
            "description" => "XXX ",
                   "from" => "XXX"
        },
        [2] {
            "received_at" => "2021-07-05T07:42:38.000+02:00",
                 "length" => 132,
                 "amount" => 0.036,
             "answer_uri" => nil,
                     "to" => "XXX",
            "description" => "XXX",
                   "from" => "XXX"
        }
    ],
           "url" => "https://httpbin.org/get",
          "args" => {},
          "time" => "2021-07-05"
}

Thank you guys so much!
Kind Regards
Marvin

You could use a split filter.

Ah yes, i completely missed the split filter in the doc. Unfortunately I can't do further actions on the fields inside the body. (It's probably a similiar behaviour to the http-poller problem from my other thread). Therefore this ain't really a solution for my usecase.

I need to access the fields to rename them. At the moment I don't know if this is even possible (with or without the split filter). The "split" documentation didn't mention anything similiar.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.