Nested JSON flattened in Logstash Filter

[Divyank_Mahalle]

Hi,
I am able to flatten at fields present in nested JSON excluding 3 fields that are present in double nested JSON. I need to flatten these fields so that I can create viz based on that such as Fields-"quantity", "rate", "period".

Also When I run a curl command for API 20 documents are getting generated instead of a single document, which is due to filter condition.

Any suggestion on filter improvement will be helpful.

Example-

{
   "costs" : {
      "data_transfer_and_storage" : 0.1,
      "resources" : 0.1,
      "total" : 0.1
   },
   "data_transfer_and_storage" : [
      {
         "cost" : 0.1,
         "name" : "string",
         "quantity" : {
            "formatted_value" : "string",
            "value" : 0
         },
         "rate" : {
            "formatted_value" : "string",
            "value" : 0.1
         },
         "sku" : "string",
         "type" : "string"
      }
   ],
   "resources" : [
      {
         "hours" : 0,
         "instance_count" : 0,
         "kind" : "string",
         "name" : "string",
         "period" : {
            "end" : "2019-01-01T00:00:00Z",
            "start" : "2019-01-01T00:00:00Z"
         },
         "price" : 0.1,
         "price_per_hour" : 0.1,
         "sku" : "string"
      }
   ]
}

My Logstash filter-

filter{

  split { field => "[resources]" }

  mutate {
    add_field => {
      "resource_price_per_hour" => "%{[resources][price_per_hour]}"
      "resource_instance_count" => "%{[resources][instance_count]}"
      "resource_name" => "%{[resources][name]}"
       "resource_sku" => "%{[resources][sku]}"
       "resource_price" => "%{[resources][price]}"
       "resource_hours" => "%{[resources][hours]}"
       "resource_kind" => "%{[resources][kind]}"

    }
    remove_field => [ "[resources]" ]
  }

   split { field => "[data_transfer_and_storage]" }

  mutate {
    add_field => {
      "data_name" => "%{[data_transfer_and_storage][name]}"
      "data_sku" => "%{[data_transfer_and_storage][sku]}"
      "data_cost" => "%{[data_transfer_and_storage][cost]}"
      "data_type" => "%{[data_transfer_and_storage][type]}"
      "rate" => "%{[data_transfer_and_storage][rate]}"
      "quantity" => "%{[data_transfer_and_storage][quantity]}"
 }
    remove_field => [ "[data_transfer_and_storage]" ]
  }

}

[Divyank_Mahalle]

Updated Filter which flattens double nested JSON-

Ref link-https://discuss.elastic.co/t/flatten-json-array-in-logstash-filter/124562

filter{

 split { field => "[resources]" }

  mutate {
    add_field => {
      "resource_price_per_hour" => "%{[resources][price_per_hour]}"
      "resource_instance_count" => "%{[resources][instance_count]}"
      "resource_name" => "%{[resources][name]}"
       "resource_sku" => "%{[resources][sku]}"
       "resource_price" => "%{[resources][price]}"
       "resource_hours" => "%{[resources][hours]}"
       "resource_kind" => "%{[resources][kind]}"
      "resource_period_start" => "%{[resources][period][start]}"
      "resource_period_end" => "%{[resources][period][end]}"

    }
    remove_field => [ "[resources]" ]
  }

   split { field => "[data_transfer_and_storage]" }

  mutate {
    add_field => {
      "data_name" => "%{[data_transfer_and_storage][name]}"
      "data_sku" => "%{[data_transfer_and_storage][sku]}"
      "data_cost" => "%{[data_transfer_and_storage][cost]}"
      "data_type" => "%{[data_transfer_and_storage][type]}"
      "data_quantity1" => "%{[data_transfer_and_storage][quantity][formatted_value]}"
      "data_quantity2" => "%{[data_transfer_and_storage][quantity][value]}"
     "data_rate1" => "%{[data_transfer_and_storage][rate][formatted_value]}"
      "data_rate2" => "%{[data_transfer_and_storage][rate][value]}"


 }
    remove_field => [ "[data_transfer_and_storage]" ]
  }
}