# Flatting nested array in Json
**URL:** https://discuss.elastic.co/t/flatting-nested-array-in-json/233066
**Category:** Logstash
**Created:** [May 18, 2020, 8:50am UTC](https://discuss.elastic.co/t/flatting-nested-array-in-json/233066 "2020-05-18T08:50:25Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@Ullas7](https://discuss.elastic.co/u/Ullas7)
#### Post date: [May 18, 2020, 8:50am UTC](https://discuss.elastic.co/t/flatting-nested-array-in-json/233066/1 "2020-05-18T08:50:25Z")
I have a Json something like this , I trying to flatten it before storing in Elasticsearch. all nested elements should be on level 1. I read about ruby script. I am new to ELK and ruby. someone can help me out?
```
{
**"school":"Someschool",
**"user":[
** {
**"name":"user1",
**"subject":[{
**"score":7,
**"topic":"science",
**"date":{"day":02,"month":04,"year":2020}
** },
** {
**"score":6,
**"topic":"Maths",
**"date":{"day":02,"month":04,"year":2020}
** }]
** },
** {
**"name":"user2",
**"subject":[{
**"score":9,
**"topic":"science",
**"date":{"day":02,"month":04,"year":2020}
** },
** {
**"score":4,
**"topic":"Maths",
**"date":{"day":02,"month":04,"year":2020}
** }]
** }
** ],
**
**"addess":[
** {"type":"present","area":"somearea","zip":12323},
** {"type":"temp","area":"somearea","zip":342}
** ],
**"reg":{"year":1990,"day":12,"month":01}
**}
```
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 18, 2020, 3:08pm UTC](https://discuss.elastic.co/t/flatting-nested-array-in-json/233066/2 "2020-05-18T15:08:28Z")
> [@Ullas7](#):
>
> all nested elements should be on level 1
What do you mean by that? If nested elements like name and subject are moved to the top level there can only be one of them, so you will lose data.
---
### Author:  [@Ullas7](https://discuss.elastic.co/u/Ullas7)
#### Post date: [May 18, 2020, 3:12pm UTC](https://discuss.elastic.co/t/flatting-nested-array-in-json/233066/3 "2020-05-18T15:12:18Z")
I would like to append index , something like this,
**{**
\*\* **"school":"Someschool",**
\*\* "user.name":"",\*\*
\*\* "user.subject.score-0":"7",\*\*
\*\* -----\*\*
\*\* "user.subject.score-1":"6"\*\*
**}**
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 18, 2020, 3:32pm UTC](https://discuss.elastic.co/t/flatting-nested-array-in-json/233066/4 "2020-05-18T15:32:27Z")
You would need to use a ruby filter. [This](https://discuss.elastic.co/t/indent-any-json-file-nested-fields-and-make-it-flat/194447/2) might help you get started. You will need to add array handling.
---
### Author:  [@Ullas7](https://discuss.elastic.co/u/Ullas7)
#### Post date: [May 18, 2020, 3:45pm UTC](https://discuss.elastic.co/t/flatting-nested-array-in-json/233066/5 "2020-05-18T15:45:20Z")
@Badger Thanks, I am already using this for JSON Object. Before that, I need to flatten array. With this link, my JSON object is getting flatten but it doesn't work on an array.
Also the above link we need to pass each field, imagine Json with ten inner JSON object we need to call ten times.
Just help me in iterating array dynamically and getting a key.
This is my plan.
1. Get all array dynamically
2. Iterate them and use the above link to flatten JSON
Repeat it for all.
I am new this, I just started exploring from last few days.
Thanks
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 18, 2020, 4:01pm UTC](https://discuss.elastic.co/t/flatting-nested-array-in-json/233066/6 "2020-05-18T16:01:35Z")
> [@Ullas7](#):
>
> Also the above link we need to pass each field, imagine Json with ten inner JSON object we need to call ten times.
No, it recursively processes the contents of hashes.
---
### Author:  [@Rahul\_Kumar4](https://discuss.elastic.co/u/Rahul_Kumar4)
#### Post date: [May 18, 2020, 9:00pm UTC](https://discuss.elastic.co/t/flatting-nested-array-in-json/233066/7 "2020-05-18T21:00:33Z")
> [@Ullas7](#):
>
> Just help me in iterating array dynamically and getting a key.
For iterating over the array dynamically, you can use the `split` filter on the field.
Split filter - [Split filter plugin | Logstash Reference [8.11] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-filters-split.html#plugins-filters-split-field)
First run the split over `user` and then another split over `subject`. This will give you one event `per user per subject` and then use @Badger's code to recursively process the hashes.
```auto
filter {
json{
source => "message"
}
split{
field => "user"
}
split{
field => "[user][subject]"
}
# code to process the hashes recursively here
}
```
---
### Author:  [@baig](https://discuss.elastic.co/u/baig)
#### Post date: [May 18, 2020, 10:24pm UTC](https://discuss.elastic.co/t/flatting-nested-array-in-json/233066/8 "2020-05-18T22:24:39Z")
What is your end goal? If you flatten this data you will lose the object relations as @Badger mentioned. Also, can you please share the mapping if you are using one?
---
### Author:  [@Claudio\_Ract\_Costa](https://discuss.elastic.co/u/Claudio_Ract_Costa)
#### Post date: [May 19, 2020, 1:19am UTC](https://discuss.elastic.co/t/flatting-nested-array-in-json/233066/9 "2020-05-19T01:19:48Z")
Ullas,
See my topic bellow where Badger helped me.
[https://discuss.elastic.co/t/split-a-json-array-with-same-fields-names/233008/12](https://discuss.elastic.co/t/split-a-json-array-with-same-fields-names/233008/12)
I think it can be used in your case.
---
### Author:  [@Ullas7](https://discuss.elastic.co/u/Ullas7)
#### Post date: [May 19, 2020, 6:58am UTC](https://discuss.elastic.co/t/flatting-nested-array-in-json/233066/10 "2020-05-19T06:58:27Z")
@Rahul_Kumar4 Thanks.
This looks simpler and easy approach. Unfortunately, there are few challenges
1. Split on multiple fields at the same level not working ( user and address, both are arrays and same level). Need to figure it out.
When I tried `field =>[user][address]` I get an error `Only String and Array types are splittable. field:[user][address] is of type = NilClass`
1. When I split array I get ` {"k":"v"}` and @Badger function needs field name to be passed ` "fn":{"k":"v"}` to flatten it .
I am still figuring out the right approach
---
### Author:  [@Ullas7](https://discuss.elastic.co/u/Ullas7)
#### Post date: [May 19, 2020, 8:50am UTC](https://discuss.elastic.co/t/flatting-nested-array-in-json/233066/11 "2020-05-19T08:50:57Z")
I just want to bring all nested keys to the first level. Technically just one { and one }.
Trying to see if it is possible to achieve. When there are two arrays it is getting complicated.
@all, when we use a split filter, does it creates two output out of one json input data? In my case, it is inserting two records with the same data except for the split keys in the elastic search.
---
### Author:  [@Ullas7](https://discuss.elastic.co/u/Ullas7)
#### Post date: [May 19, 2020, 1:09pm UTC](https://discuss.elastic.co/t/flatting-nested-array-in-json/233066/12 "2020-05-19T13:09:19Z")
@Badger Yeah it processes recursively on nested json. I ment parallel json Object. Lets say
```auto
{
"A": {"A1":"VA1","A2":"VA2"},
"B": {"B1":"VB1","B2":"VB2"}
}
```
In this case I need to call twice with A and B. I made small change in the ruby script (from your link) and able to process by sending comma separated values Sharing for future readers
```auto
def register(params)
@field = params['field']
end
def flatten(object, name, event)
if object
if object.kind_of?(Hash) || object == {} || object==[]
object.each { |k, v| flatten(v, "#{name}.#{k}", event) }
else
event.set(name, object)
end
end
end
def filter(event)
inputs = @field.split(',')
for input in inputs
o = event.get(input)
if o
flatten(o, input, event)
end
event.remove(input)
end
[event]
end
```
Thank you for a wonderful and quick response.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 16, 2020, 1:09pm UTC](https://discuss.elastic.co/t/flatting-nested-array-in-json/233066/13 "2020-06-16T13:09:21Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.