# How to create fields from nested fields for json data in logstash

**URL:** <https://discuss.elastic.co/t/how-to-create-fields-from-nested-fields-for-json-data-in-logstash/239369>\
**Category:** Logstash\
**Created:** [June 30, 2020, 7:24pm UTC](https://discuss.elastic.co/t/how-to-create-fields-from-nested-fields-for-json-data-in-logstash/239369 "2020-06-30T19:24:23Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ramalakshmi](https://avatars.discourse-cdn.com/v4/letter/r/f9ae1b/32.png) [@Ramalakshmi](https://discuss.elastic.co/u/Ramalakshmi)\
**Post date:** [June 30, 2020, 7:24pm UTC](https://discuss.elastic.co/t/how-to-create-fields-from-nested-fields-for-json-data-in-logstash/239369/1 "2020-06-30T19:24:24Z")

</div>

Hi,  
I am having json data that is parsed into kibana using logstash, in that data I am getting nested fields but I want individual fields.Please tell me how to get those fields.

```
service.action.portProbeAction.portProbeDetails {
  "localPortDetails": {
    "portName": "HTTPS",
    "port": 443
  },
  "remoteIpDetails": {
    "organization": {
      "org": " ",
      "asnOrg": " ",
      "asn": "49505",
      "isp": " "
    },
    "city": {
      "cityName": ""
    },
    "country": {
      "countryName": "us"
    },
    "geoLocation": {
      "lat": 20.7386,
      "lon": -56.6068
    },
    "ipAddressV4": "76.98.213,56"
  }
},
{
  "localPortDetails": {
    "portName": "HTTP",
    "port": 80
  },
  "remoteIpDetails": {
    "organization": {
      "org": " ",
      "asnOrg": " ",
      "asn": "49505",
      "isp": " "
    },
    "city": {
      "cityName": ""
    },
    "country": {
      "countryName": "Russia"
    },
    "geoLocation": {
      "lat": 21.7386,
      "lon": 8.6068
    },
    "ipAddressV4": 76.98.213.56"

```

expected output fileds service.action.portProbeAction.portProbeDetailsremoteipDetails.city.contryname\_1: US  
service.action.portProbeAction.portProbeDetailsremoteipDetails.city.contryname\_2:Russia

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 30, 2020, 8:11pm UTC](https://discuss.elastic.co/t/how-to-create-fields-from-nested-fields-for-json-data-in-logstash/239369/2 "2020-06-30T20:11:35Z")

</div>

That is not valid JSON. Is service.action.portProbeAction.portProbeDetails an array?

---

<div class="post-metadata">

**Author:** ![Ramalakshmi](https://avatars.discourse-cdn.com/v4/letter/r/f9ae1b/32.png) [@Ramalakshmi](https://discuss.elastic.co/u/Ramalakshmi)\
**Post date:** [June 30, 2020, 8:25pm UTC](https://discuss.elastic.co/t/how-to-create-fields-from-nested-fields-for-json-data-in-logstash/239369/3 "2020-06-30T20:25:47Z")

</div>

yes,I have posted only nested field not entire json data.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 30, 2020, 8:42pm UTC](https://discuss.elastic.co/t/how-to-create-fields-from-nested-fields-for-json-data-in-logstash/239369/4 "2020-06-30T20:42:35Z")

</div>

It is not quite what you asked for, but should give you something to work with

```
    ruby {
        code => '
            def flattenObject(object, name, event)
                if object
                    if object.kind_of?(Hash) and object != {}
                        object.each { |k, v| flattenObject(v, "#{name}.#{k}", event) }
                    elsif object.kind_of?(Array) and object != []
                        object.each_index { |i|
                            flattenObject(object[i], name + "_#{i}", event)
                        }
                    else
                        event.set(name, object)
                    end
                end
            end

            fieldName = "service.action.portProbeAction.portProbeDetails"
            o = event.get(fieldName)
            if o
                flattenObject(o, fieldName, event)
            end
            event.remove(fieldName)
        '
    }

```

will produce

```
 "service.action.portProbeAction.portProbeDetails_0.remoteIpDetails.organization.asnOrg" => " ",
    "service.action.portProbeAction.portProbeDetails_0.remoteIpDetails.geoLocation.lat" => 20.7386,
"service.action.portProbeAction.portProbeDetails_1.remoteIpDetails.country.countryName" => "Russia",
   "service.action.portProbeAction.portProbeDetails_1.remoteIpDetails.organization.org" => " ",
          "service.action.portProbeAction.portProbeDetails_0.localPortDetails.portName" => "HTTPS",
   "service.action.portProbeAction.portProbeDetails_0.remoteIpDetails.organization.isp" => " ",
"service.action.portProbeAction.portProbeDetails_0.remoteIpDetails.country.countryName" => "us",
    "service.action.portProbeAction.portProbeDetails_1.remoteIpDetails.geoLocation.lon" => 8.6068,
    "service.action.portProbeAction.portProbeDetails_0.remoteIpDetails.geoLocation.lon" => -56.6068,
      "service.action.portProbeAction.portProbeDetails_0.remoteIpDetails.city.cityName" => "",
   "service.action.portProbeAction.portProbeDetails_1.remoteIpDetails.organization.isp" => " ",
   "service.action.portProbeAction.portProbeDetails_0.remoteIpDetails.organization.org" => " ",
"service.action.portProbeAction.portProbeDetails_1.remoteIpDetails.organization.asnOrg" => " ",
          "service.action.portProbeAction.portProbeDetails_1.localPortDetails.portName" => "HTTP",
    "service.action.portProbeAction.portProbeDetails_1.remoteIpDetails.geoLocation.lat" => 21.7386,
      "service.action.portProbeAction.portProbeDetails_1.remoteIpDetails.city.cityName" => "",
   "service.action.portProbeAction.portProbeDetails_0.remoteIpDetails.organization.asn" => "49505",
        "service.action.portProbeAction.portProbeDetails_1.remoteIpDetails.ipAddressV4" => "76.98.213.56",
              "service.action.portProbeAction.portProbeDetails_1.localPortDetails.port" => 80,
   "service.action.portProbeAction.portProbeDetails_1.remoteIpDetails.organization.asn" => "49505",
        "service.action.portProbeAction.portProbeDetails_0.remoteIpDetails.ipAddressV4" => "76.98.213.56",
              "service.action.portProbeAction.portProbeDetails_0.localPortDetails.port" => 443,
```

---

<div class="post-metadata">

**Author:** ![Ramalakshmi](https://avatars.discourse-cdn.com/v4/letter/r/f9ae1b/32.png) [@Ramalakshmi](https://discuss.elastic.co/u/Ramalakshmi)\
**Post date:** [June 30, 2020, 8:56pm UTC](https://discuss.elastic.co/t/how-to-create-fields-from-nested-fields-for-json-data-in-logstash/239369/5 "2020-06-30T20:56:19Z")

</div>

ThaKyou.  
But I want filed name as  
"service.action.portProbeAction.portProbeDetails.remoteIpDetails.organization.city.cityname\_1" =\> " "  
And also I am getting two more nested filed in the message so, the above code can I use that

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 30, 2020, 9:12pm UTC](https://discuss.elastic.co/t/how-to-create-fields-from-nested-fields-for-json-data-in-logstash/239369/6 "2020-06-30T21:12:27Z")

</div>

> [@Ramalakshmi](#):
>
> But I want filed name as  
> "service.action.portProbeAction.portProbeDetails.remoteIpDetails.organization.city.cityname\_1" =\> " "

I realize that. Feel free to modify the code to get that.

---

<div class="post-metadata">

**Author:** ![Ramalakshmi](https://avatars.discourse-cdn.com/v4/letter/r/f9ae1b/32.png) [@Ramalakshmi](https://discuss.elastic.co/u/Ramalakshmi)\
**Post date:** [July 1, 2020, 11:47am UTC](https://discuss.elastic.co/t/how-to-create-fields-from-nested-fields-for-json-data-in-logstash/239369/7 "2020-07-01T11:47:06Z")

</div>

While running ruby code , I am not getting same output what I posted. There is no change in the output.

---

<div class="post-metadata">

**Author:** ![Ramalakshmi](https://avatars.discourse-cdn.com/v4/letter/r/f9ae1b/32.png) [@Ramalakshmi](https://discuss.elastic.co/u/Ramalakshmi)\
**Post date:** [July 1, 2020, 8:08pm UTC](https://discuss.elastic.co/t/how-to-create-fields-from-nested-fields-for-json-data-in-logstash/239369/8 "2020-07-01T20:08:16Z")

</div>

Please help out for this code. I am not getting output what I expected.

---

<div class="post-metadata">

**Author:** ![Ramalakshmi](https://avatars.discourse-cdn.com/v4/letter/r/f9ae1b/32.png) [@Ramalakshmi](https://discuss.elastic.co/u/Ramalakshmi)\
**Post date:** [July 3, 2020, 4:12pm UTC](https://discuss.elastic.co/t/how-to-create-fields-from-nested-fields-for-json-data-in-logstash/239369/9 "2020-07-03T16:12:33Z")

</div>

I have tried with the ruby code in logstash but not worked for me.Please help me.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 31, 2020, 4:18pm UTC](https://discuss.elastic.co/t/how-to-create-fields-from-nested-fields-for-json-data-in-logstash/239369/10 "2020-07-31T16:18:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.