# Nested JSON use field as value

**URL:** <https://discuss.elastic.co/t/nested-json-use-field-as-value/116831>\
**Category:** Logstash\
**Created:** [January 24, 2018, 9:02am UTC](https://discuss.elastic.co/t/nested-json-use-field-as-value/116831 "2018-01-24T09:02:21Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![zacesa](https://avatars.discourse-cdn.com/v4/letter/z/d07c76/32.png) [@zacesa](https://discuss.elastic.co/u/zacesa)\
**Post date:** [January 24, 2018, 9:02am UTC](https://discuss.elastic.co/t/nested-json-use-field-as-value/116831/1 "2018-01-24T09:02:21Z")

</div>

Hi,

I have a json input as per below;

```
"stats": {
"src": {
  "port": {
    "6666": 1
  },
  "rep": 1,
  "country": {
    "US": 1
  },
  "ip": {
    "1.1.1.1": {
      "rep": 1,
      "uuid": "NULL",
      "country": "US",
      "count": 1
    }
  }
},
"dst": {
  "port": {
    "443": 1
  },
  "rep": 0,
  "country": {
    "US": 1
  },
  "ip": {
    "2.2.2.2": {
      "rep": 0,
      "uuid": "0x29bfa15ae6da11e7a1ac000cfb542710",
      "country": "US",
      "count": 1
    }
  }
},
"events": 1
  },

```

I'd like to do a geoip of the [src][ip] however, the value of [src][ip] is in fact, "1.1.1.1": { "rep": 1, "uuid": "NULL", "country": "US", "count": 1 }.

I tried to copy [src][ip] to a new field called src\_ip but, it copies the whole thing. Is there any way to remove the other fields under src\_ip?

I've already tried,

```
 mutate {
        remove_field => ["[src_ip][rep]", "[src_ip][uuid]", "[src_ip][count]", "[src_ip][country]" ]
    }

```

but it didn't work.

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [January 24, 2018, 4:35pm UTC](https://discuss.elastic.co/t/nested-json-use-field-as-value/116831/2 "2018-01-24T16:35:31Z")

</div>

Unfortunately, we do not have a mechanism to refer to an anonymous key in the nested object. That is, we don't have `[src][ip^K:1]` as in "grab the first key name of the nested object in `[src][ip]`. Obviously `[src][ip][1.1.1.1]` is a reference to the nested object `{"rep": 1, ... "count": 1}`

You could try to transform the JSON before its pushed into LS. [https://github.com/ColinEberhardt/json-transforms](https://github.com/ColinEberhardt/json-transforms) is a possible OS project.

---

<div class="post-metadata">

**Author:** ![zacesa](https://avatars.discourse-cdn.com/v4/letter/z/d07c76/32.png) [@zacesa](https://discuss.elastic.co/u/zacesa)\
**Post date:** [January 25, 2018, 3:00am UTC](https://discuss.elastic.co/t/nested-json-use-field-as-value/116831/3 "2018-01-25T03:00:43Z")

</div>

I've gone with another method to get the data into a field.

I used;

> grok {  
> match =\> { "stats" =\> "\"src\"\:\{.\*?\"ip\"\:\{.\*?\"%{IP:src\_ip}\"" }  
> }

However, there may be multiple source IP so, how do I use the same grok pattern, to get multiple matches on the same line?

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [January 25, 2018, 11:55am UTC](https://discuss.elastic.co/t/nested-json-use-field-as-value/116831/4 "2018-01-25T11:55:22Z")

</div>

Nice idea. This done before the JSON filter, I presume?

You can have multiple patterns if the match value is an array and `break_on_match` is `false`, the src\_ip will become an array of the IPs extracted as a root key and value. Later you can add that array back into the nested hash/object. P.S. you can use single quote to avoid the escaping of double quotes.

I could only extract 3 IP addresses though (first, second & last)

```auto
input {
  generator {
    message => '{
  "src": {
    "port": {
      "6666": 1
    },
    "rep": 1,
    "country": {
      "US": 1
    },
    "ip": {
      "1.1.1.1": {
        "rep": 1,
        "uuid": "NULL",
        "country": "US",
        "count": 1
      },
      "2.1.1.1": {
        "rep": 1,
        "uuid": "NULL",
        "country": "US",
        "count": 1
      },
      "3.1.1.1": {
        "rep": 1,
        "uuid": "NULL",
        "country": "US",
        "count": 1
      }
    }
  }
}'
    count => 1
  }
}

filter {
  grok {
    match => { "message" => ['{[^"]+"%{IP:src_ip}"', '},[^"]+"%{IP:src_ip}":\s{[^}]+},\n', '},[^"]+"%{IP:src_ip}":\s{[^}]+}\n'] }
    break_on_match => false
  }
  json {
    source => "message"
    remove_field => ["message"]
  }
}

output {
  stdout {
    codec => rubydebug
  }
}

```

Results:

```auto
{
    "@timestamp" => 2018-01-25T11:25:19.920Z,
           "src" => {
           "port" => {
            "6666" => 1
        },
            "rep" => 1,
             "ip" => {
            "3.1.1.1" => {
                  "count" => 1,
                   "uuid" => "NULL",
                    "rep" => 1,
                "country" => "US"
            },
            "1.1.1.1" => {
                  "count" => 1,
                   "uuid" => "NULL",
                    "rep" => 1,
                "country" => "US"
            },
            "2.1.1.1" => {
                  "count" => 1,
                   "uuid" => "NULL",
                    "rep" => 1,
                "country" => "US"
            }
        },
        "country" => {
            "US" => 1
        }
    },
          "host" => "Elastics-MacBook-Pro.local",
      "@version" => "1",
      "sequence" => 0,
        "src_ip" => "1.1.1.1"
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 22, 2018, 11:55am UTC](https://discuss.elastic.co/t/nested-json-use-field-as-value/116831/5 "2018-02-22T11:55:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.