# Logatash with nested Json parser

**URL:** <https://discuss.elastic.co/t/logatash-with-nested-json-parser/41198>\
**Category:** Logstash\
**Created:** [February 8, 2016, 3:50pm UTC](https://discuss.elastic.co/t/logatash-with-nested-json-parser/41198 "2016-02-08T15:50:59Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![vimalmpatel](https://avatars.discourse-cdn.com/v4/letter/v/49beb7/32.png) [@vimalmpatel](https://discuss.elastic.co/u/vimalmpatel)\
**Post date:** [February 8, 2016, 3:51pm UTC](https://discuss.elastic.co/t/logatash-with-nested-json-parser/41198/1 "2016-02-08T15:51:00Z")

</div>

i have kafka input

something like this

{"interactionId":"7891013","applicantInfoList":[{"cif":123456,"role":"Primary","applicantType":"PROSPECT"}],"correlationId":"d828bcf9-0fb7-467b-b2c5-a67c2305a9db","applicationId":"99999","productType":4600,"timestamp":1443713482677}

logstash configuration

input {  
file {  
path =\> "/Users/xxx/Desktop/ELK/ELK Infrastructure/kafka\_xxx\_sample\_data.txt"  
start\_position =\> "beginning"  
}  
}

filter {  
json {  
source =\> "message"  
}  
if [applicantInfoList][cif] == 123456 {  
mutate {  
add\_field =\> {  
"new\_field" =\> "Application Accepted"  
}  
}  
}  
else if [applicantInfoList][cif] == 7891011 {  
mutate {  
add\_field =\> {  
"new\_field" =\> "Application Rejected"  
}  
}  
}  
}

output {  
elasticsearch {  
action =\> "index"  
hosts =\> "127.0.0.1:9200"  
codec =\> "plain"  
flush\_size =\> 5000  
idle\_flush\_time =\> 1  
index =\> "logstash-%{+YYYY.MM.dd}"  
}  
stdout { codec =\> rubydebug }  
}

Issue : it is not recognizing nested field [applicantInfoList][cif] because of []

but if i remove [] from input it is able to recognize nested field cif.

how would i solve it without removing it [].

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 9, 2016, 7:33pm UTC](https://discuss.elastic.co/t/logatash-with-nested-json-parser/41198/2 "2016-02-09T19:33:25Z")

</div>

`applicantInfoList` contains an array of objects so `[applicantInfoList][cif]` won't work. You'd have to indicate _which_ of the elements in the array you want to access. For example, the first element should be accessible via `[applicantInfoList][0][cif]`.

---

<div class="post-metadata">

**Author:** ![vimalmpatel](https://avatars.discourse-cdn.com/v4/letter/v/49beb7/32.png) [@vimalmpatel](https://discuss.elastic.co/u/vimalmpatel)\
**Post date:** [February 11, 2016, 1:56pm UTC](https://discuss.elastic.co/t/logatash-with-nested-json-parser/41198/3 "2016-02-11T13:56:41Z")

</div>

Thanks

---

<div class="post-metadata">

**Author:** ![vimalmpatel](https://avatars.discourse-cdn.com/v4/letter/v/49beb7/32.png) [@vimalmpatel](https://discuss.elastic.co/u/vimalmpatel)\
**Post date:** [February 11, 2016, 7:30pm UTC](https://discuss.elastic.co/t/logatash-with-nested-json-parser/41198/4 "2016-02-11T19:30:24Z")

</div>

Hi

example

{"interactionId":"7891013","applicantInfoList":[{"cif":123456,"role":"Primary","applicantType":"PROSPECT"},{"cif":123456,"role":"Primary","applicantType":"PROSPECT"},{"cif":123456,"role":"Primary","applicantType":"PROSPECT"}],"correlationId":"d828bcf9-0fb7-467b-b2c5-a67c2305a9db","applicationId":"99999","productType":4600,"timestamp":1443713482677}

{"interactionId":"7891013","applicantInfoList":[{"cif":123456,"role":"Primary","applicantType":"PROSPECT"},{"cif":123456,"role":"Primary","applicantType":"PROSPECT"}],"correlationId":"d828bcf9-0fb7-467b-b2c5-a67c2305a9db","applicationId":"99999","productType":4600,"timestamp":1443713482677}

it could be two, three or unknown nested data.

if i have repetitive nested data is there a way to do it dynamically instead of hardcoding [applicantInfoList][0][cif] or [applicantInfoList][1][cif] ......?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 11, 2016, 7:47pm UTC](https://discuss.elastic.co/t/logatash-with-nested-json-parser/41198/5 "2016-02-11T19:47:04Z")

</div>

Yes, but you'll have to use a ruby filter. I'm afraid I don't have time to write up an example.

---

<div class="post-metadata">

**Author:** ![vimalmpatel](https://avatars.discourse-cdn.com/v4/letter/v/49beb7/32.png) [@vimalmpatel](https://discuss.elastic.co/u/vimalmpatel)\
**Post date:** [February 17, 2016, 7:40pm UTC](https://discuss.elastic.co/t/logatash-with-nested-json-parser/41198/6 "2016-02-17T19:40:58Z")

</div>

i not able to figure out deep nested json

example :

{"interactionId":"7891013","applicantInfoList":[{"pri\_cif":123456,"role":"Primary","applicantType":"PROSPECT"},{"sec\_cif":123456,"role":"secondary","applicantType":"PROSPECT","homeAddress" =\> {  
"addressType" =\> "RESIDENCE",  
"addressLineOne" =\> "3715 W. Trevino",  
"addressLineTwo" =\> "a",  
"city" =\> "Hobbs",  
"state" =\> "NM",  
"zip" =\> "88240",  
"country" =\> "US"  
}}],"correlationId":"d828bcf9-0fb7-467b-b2c5-a67c2305a9db","applicationId":"99999","productType":4600,"timestamp":1443713482677}

this is not working

filter {  
ruby {  
code =\> "event['applicantInfoList'][homeAddress].each { |x| x.delete('addressType') .each { |x| x.delete('addressLineOne')}"  
}  
}

any suggestion on how to delete deep nested fields ?

---

<div class="post-metadata">

**Author:** ![vimalmpatel](https://avatars.discourse-cdn.com/v4/letter/v/49beb7/32.png) [@vimalmpatel](https://discuss.elastic.co/u/vimalmpatel)\
**Post date:** [February 17, 2016, 9:33pm UTC](https://discuss.elastic.co/t/logatash-with-nested-json-parser/41198/7 "2016-02-17T21:33:43Z")

</div>

i used this and worked fine

filter {  
ruby {  
code =\> "event['applicantInfoList'].each { |x| if x['homeAddress']; x['homeAddress'].delete('addressType'); x['homeAddress'].delete('addressLineOne') end }"  
}  
}

---

<div class="post-metadata">

**Author:** ![Joe\_Lawson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joe_lawson/32/3390_2.png) [@Joe\_Lawson](https://discuss.elastic.co/u/Joe_Lawson)\
**Post date:** [February 18, 2016, 5:29pm UTC](https://discuss.elastic.co/t/logatash-with-nested-json-parser/41198/8 "2016-02-18T17:29:59Z")

</div>

Thanks for replying with you example! 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:10am UTC](https://discuss.elastic.co/t/logatash-with-nested-json-parser/41198/9 "2017-07-06T05:10:51Z")

</div>