# Parsing json data which has list of arrays inside array

**URL:** <https://discuss.elastic.co/t/parsing-json-data-which-has-list-of-arrays-inside-array/124080>\
**Category:** Logstash\
**Created:** [March 15, 2018, 11:12am UTC](https://discuss.elastic.co/t/parsing-json-data-which-has-list-of-arrays-inside-array/124080 "2018-03-15T11:12:58Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![kumarkar](https://avatars.discourse-cdn.com/v4/letter/k/57b2e6/32.png) [@kumarkar](https://discuss.elastic.co/u/kumarkar)\
**Post date:** [March 15, 2018, 11:12am UTC](https://discuss.elastic.co/t/parsing-json-data-which-has-list-of-arrays-inside-array/124080/1 "2018-03-15T11:12:58Z")

</div>

Hi All,

I am new to logstash, i have a requirement to parse the huge json file. Can someone help me out here?  
Is it possible to parse the below content using grok script?  
Below is the sample file content:  
{  
"environments": [ {  
"dimensions": [  
{  
"metrics": [  
{  
"name": "sum(target\_response\_time)",  
"values": [ {  
"timestamp": 1520463600000,  
"value": "146.0"  
}]  
},  
{  
"name": " sum(total\_response\_time)",  
"values": [ {  
"timestamp": 1520463600000,  
"value": "170.0"  
}]  
},  
{  
"name": "sum(response\_size)",  
"values": [ {  
"timestamp": 1520463600000,  
"value": "108.0"  
}]  
}  
],  
"name": "66504ed5-0378-d1b5-c95e-abd6a2ef6d45"  
},  
{  
"metrics": [  
{  
"name": "sum(target\_response\_time)",  
"values": [ {  
"timestamp": 1520463600000,  
"value": "221.0"  
}]  
},  
{  
"name": " sum(total\_response\_time)",  
"values": [ {  
"timestamp": 1520463600000,  
"value": "222.0"  
}]  
},  
{  
"name": "sum(response\_size)",  
"values": [ {  
"timestamp": 1520463600000,  
"value": "168.0"  
}]  
}  
],  
"name": "3d651b1e-fce3-0b31-c128-f4b245fddda5"  
}  
],  
"name": "dev"  
}],  
"metaData": {  
"errors": [],  
"notices": [  
"source pg:33bd822f-967b-4454-9572-29257cb112f8",  
]  
}  
}

Any quick help is appreciated.

---

<div class="post-metadata">

**Author:** ![Krunal\_kalaria](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/krunal_kalaria/32/23862_2.png) [@Krunal\_kalaria](https://discuss.elastic.co/u/Krunal_kalaria)\
**Post date:** [March 15, 2018, 12:12pm UTC](https://discuss.elastic.co/t/parsing-json-data-which-has-list-of-arrays-inside-array/124080/2 "2018-03-15T12:12:46Z")

</div>

Hi @kumarkar,

Why you need to write the grok ?  
In filter plugin already we have json filter it will parse the json data just refer this link:

[https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html](https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html)

otherwise if you have any file then try this little example it may work:

input  
{  
file  
{  
path =\> "C:\Test\file.json" # your file path  
type =\> "json"  
sincedb\_path =\> "dev/null"  
start\_position =\> "beginning"  
codec =\> "json"  
}  
}

filter  
{  
json  
{  
source =\> "message"  
}

}

output  
{  
stdout { codec =\> rubydebug }  
elasticsearch  
{  
hosts =\> ["localhost:9200"]  
index =\> "test-%{+YYYY.MM.dd}"

```
            }

```

}

Thanks & Regard,  
Krunal.

---

<div class="post-metadata">

**Author:** ![kumarkar](https://avatars.discourse-cdn.com/v4/letter/k/57b2e6/32.png) [@kumarkar](https://discuss.elastic.co/u/kumarkar)\
**Post date:** [March 15, 2018, 1:49pm UTC](https://discuss.elastic.co/t/parsing-json-data-which-has-list-of-arrays-inside-array/124080/3 "2018-03-15T13:49:50Z")

</div>

Hi @Krunal_kalaria

I tried the way you specified and outputing to file, i see it is trying to parse line by line and failing, below is the few line of output file.  
{"path":"D:\ELK\Test\test.json","tags":["\_jsonparsefailure"],"message":"{\r","type":"json","@version":"1","host":"A2ML20110","@timestamp":"2018-03-15T12:40:13.888Z"}  
{"path":"D:\ELK\Test\test.json","tags":["\_jsonparsefailure"],"message":" "environments": [ {\r","type":"json","@version":"1","host":"A2ML20110","@timestamp":"2018-03-15T12:40:13.945Z"}  
{"path":"D:\ELK\Test\test.json","tags":["\_jsonparsefailure"],"message":" "dimensions": [\r","type":"json","@version":"1","host":"A2ML20110","@timestamp":"2018-03-15T12:40:13.956Z"}  
{"path":"D:\ELK\Test\test.json","tags":["\_jsonparsefailure"],"message":" {\r","type":"json","@version":"1","host":"A2ML20110","@timestamp":"2018-03-15T12:40:13.972Z"}  
{"path":"D:\ELK\Test\test.json","tags":["\_jsonparsefailure"],"message":" "name": " sum(total\_response\_time)",\r","type":"json","@version":"1","host":"A2ML20110","@timestamp":"2018-03-15T12:40:14.305Z"}  
{"path":"D:\ELK\Test\test.json","tags":["\_jsonparsefailure"],"message":" "values": [ {\r","type":"json","@version":"1","host":"A2ML20110","@timestamp":"2018-03-15T12:40:14.371Z"}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 18, 2018, 8:10pm UTC](https://discuss.elastic.co/t/parsing-json-data-which-has-list-of-arrays-inside-array/124080/4 "2018-03-18T20:10:41Z")

</div>

Use a multiline codec to join the lines of the file to a single event (you should be able to find a working example in previous posts), then use a json filter to parse the resulting JSON string.

---

<div class="post-metadata">

**Author:** ![kumarkar](https://avatars.discourse-cdn.com/v4/letter/k/57b2e6/32.png) [@kumarkar](https://discuss.elastic.co/u/kumarkar)\
**Post date:** [March 20, 2018, 4:44am UTC](https://discuss.elastic.co/t/parsing-json-data-which-has-list-of-arrays-inside-array/124080/5 "2018-03-20T04:44:05Z")

</div>

Hi @magnusbaeck,

i tried with the below, but it is not parsing the file, if possible can you please provide me some sample code:

input  
{  
file  
{  
path =\> "D:\ELK\Test\test.json"  
codec =\> multiline {  
pattern =\> "^}"  
negate =\> true  
what =\> next  
max\_lines =\> 20000  
}  
}  
}

filter{  
mutate { gsub =\> ["message", "\n", ""] }  
json{ source =\> "message" }  
split{ field =\> "Records" }  
}

output  
{  
stdout { codec =\> rubydebug }  
file  
{  
path =\> "D:\ELK\Test\Out.json"  
create\_if\_deleted =\> "true"  
flush\_interval =\> "0"  
}  
}

Regards,  
Ravi.

---

<div class="post-metadata">

**Author:** ![kumarkar](https://avatars.discourse-cdn.com/v4/letter/k/57b2e6/32.png) [@kumarkar](https://discuss.elastic.co/u/kumarkar)\
**Post date:** [April 3, 2018, 5:51am UTC](https://discuss.elastic.co/t/parsing-json-data-which-has-list-of-arrays-inside-array/124080/6 "2018-04-03T05:51:16Z")

</div>

can someone help me here?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 1, 2018, 5:51am UTC](https://discuss.elastic.co/t/parsing-json-data-which-has-list-of-arrays-inside-array/124080/7 "2018-05-01T05:51:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.