# Nested JSON flattened in Logstash Filter

**URL:** <https://discuss.elastic.co/t/nested-json-flattened-in-logstash-filter/281502>\
**Category:** Logstash\
**Created:** [August 16, 2021, 5:57am UTC](https://discuss.elastic.co/t/nested-json-flattened-in-logstash-filter/281502 "2021-08-16T05:57:41Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Divyank\_Mahalle](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/divyank_mahalle/32/91240_2.png) [@Divyank\_Mahalle](https://discuss.elastic.co/u/Divyank_Mahalle)\
**Post date:** [August 16, 2021, 5:57am UTC](https://discuss.elastic.co/t/nested-json-flattened-in-logstash-filter/281502/1 "2021-08-16T05:57:41Z")

</div>

Hi,  
I am able to flatten at fields present in nested JSON excluding 3 fields that are present in double nested JSON. I need to flatten these fields so that I can create viz based on that such as Fields-"quantity", "rate", "period".

Also When I run a curl command for API 20 documents are getting generated instead of a single document, which is due to filter condition.

Any suggestion on filter improvement will be helpful.

Example-

```auto
{
   "costs" : {
      "data_transfer_and_storage" : 0.1,
      "resources" : 0.1,
      "total" : 0.1
   },
   "data_transfer_and_storage" : [
      {
         "cost" : 0.1,
         "name" : "string",
         "quantity" : {
            "formatted_value" : "string",
            "value" : 0
         },
         "rate" : {
            "formatted_value" : "string",
            "value" : 0.1
         },
         "sku" : "string",
         "type" : "string"
      }
   ],
   "resources" : [
      {
         "hours" : 0,
         "instance_count" : 0,
         "kind" : "string",
         "name" : "string",
         "period" : {
            "end" : "2019-01-01T00:00:00Z",
            "start" : "2019-01-01T00:00:00Z"
         },
         "price" : 0.1,
         "price_per_hour" : 0.1,
         "sku" : "string"
      }
   ]
}

```

My Logstash filter-

```auto
filter{

  split { field => "[resources]" }

  mutate {
    add_field => {
      "resource_price_per_hour" => "%{[resources][price_per_hour]}"
      "resource_instance_count" => "%{[resources][instance_count]}"
      "resource_name" => "%{[resources][name]}"
       "resource_sku" => "%{[resources][sku]}"
       "resource_price" => "%{[resources][price]}"
       "resource_hours" => "%{[resources][hours]}"
       "resource_kind" => "%{[resources][kind]}"

    }
    remove_field => ["[resources]" ]
  }

   split { field => "[data_transfer_and_storage]" }

  mutate {
    add_field => {
      "data_name" => "%{[data_transfer_and_storage][name]}"
      "data_sku" => "%{[data_transfer_and_storage][sku]}"
      "data_cost" => "%{[data_transfer_and_storage][cost]}"
      "data_type" => "%{[data_transfer_and_storage][type]}"
      "rate" => "%{[data_transfer_and_storage][rate]}"
      "quantity" => "%{[data_transfer_and_storage][quantity]}"
 }
    remove_field => ["[data_transfer_and_storage]" ]
  }

}

```

---

<div class="post-metadata">

**Author:** ![Divyank\_Mahalle](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/divyank_mahalle/32/91240_2.png) [@Divyank\_Mahalle](https://discuss.elastic.co/u/Divyank_Mahalle)\
**Post date:** [August 16, 2021, 8:11am UTC](https://discuss.elastic.co/t/nested-json-flattened-in-logstash-filter/281502/2 "2021-08-16T08:11:47Z")

</div>

Updated Filter which flattens double nested JSON-

Ref link-[https://discuss.elastic.co/t/flatten-json-array-in-logstash-filter/124562](https://discuss.elastic.co/t/flatten-json-array-in-logstash-filter/124562)

```auto
filter{

 split { field => "[resources]" }

  mutate {
    add_field => {
      "resource_price_per_hour" => "%{[resources][price_per_hour]}"
      "resource_instance_count" => "%{[resources][instance_count]}"
      "resource_name" => "%{[resources][name]}"
       "resource_sku" => "%{[resources][sku]}"
       "resource_price" => "%{[resources][price]}"
       "resource_hours" => "%{[resources][hours]}"
       "resource_kind" => "%{[resources][kind]}"
      "resource_period_start" => "%{[resources][period][start]}"
      "resource_period_end" => "%{[resources][period][end]}"

    }
    remove_field => ["[resources]" ]
  }

   split { field => "[data_transfer_and_storage]" }

  mutate {
    add_field => {
      "data_name" => "%{[data_transfer_and_storage][name]}"
      "data_sku" => "%{[data_transfer_and_storage][sku]}"
      "data_cost" => "%{[data_transfer_and_storage][cost]}"
      "data_type" => "%{[data_transfer_and_storage][type]}"
      "data_quantity1" => "%{[data_transfer_and_storage][quantity][formatted_value]}"
      "data_quantity2" => "%{[data_transfer_and_storage][quantity][value]}"
     "data_rate1" => "%{[data_transfer_and_storage][rate][formatted_value]}"
      "data_rate2" => "%{[data_transfer_and_storage][rate][value]}"

 }
    remove_field => ["[data_transfer_and_storage]" ]
  }
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 13, 2021, 8:12am UTC](https://discuss.elastic.co/t/nested-json-flattened-in-logstash-filter/281502/3 "2021-09-13T08:12:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.