# \[SOLVED\] Split filter question a.k.a flatten json sub array
**URL:** https://discuss.elastic.co/t/solved-split-filter-question-a-k-a-flatten-json-sub-array/130481
**Category:** Logstash
**Created:** [May 3, 2018, 2:53pm UTC](https://discuss.elastic.co/t/solved-split-filter-question-a-k-a-flatten-json-sub-array/130481 "2018-05-03T14:53:55Z")
**Posts on this page:** 16
**Page:** 1
### Author:  [@blastik](https://discuss.elastic.co/u/blastik)
#### Post date: [May 3, 2018, 2:53pm UTC](https://discuss.elastic.co/t/solved-split-filter-question-a-k-a-flatten-json-sub-array/130481/1 "2018-05-03T14:53:55Z")
Hi there!
I have a json input which looks like this:
> [{
> "Action": "COUNT",
> "Timestamp": "2018-05-02T13:09:58Z",
> "Request": {
> "Country": "ES",
> "URI": "/uploadMultiplePhotos.aspx",
> "Headers": [{
> **\> "Name": "Host",**
> **\> "Value": "[www.example.net](http://www.example.net)"**
> }, {
> "Name": "Content-Length",
> "Value": "226245"
> }, {
> "Name": "origin",
> "Value": "[https://www.example.net](https://www.example.net)"
> }, {
> "Name": "user-agent",
> "Value": "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36"
> }, {
> "Name": "content-type",
> "Value": "multipart/form-data; boundary=----WebKitFormBoundaryBnMGtRJgyfhSDZt3"
> }, {
> "Name": "accept",
> "Value": "application/json"
> }, {
> "Name": "cache-control",
> "Value": "no-cache"
> }, {
> "Name": "x-requested-with",
> "Value": "XMLHttpRequest"
> }, {
> "Name": "x-ajax",
> "Value": "example"
> }, {
> "Name": "referer",
> "Value": "[https://www.example.net/bla.aspx](https://www.example.net/bla.aspx)"
> }, {
> "Name": "accept-encoding",
> "Value": "gzip, deflate, br"
> }, {
> "Name": "accept-language",
> "Value": "es-ES,es;q=0.8"
> }, {
> "Name": "cookie",
> "Value": "GREEDY"
> }],
> "ClientIP": "8.8.8.8",
> "Method": "POST",
> "HTTPVersion": "HTTP/2.0"
> },
> "Weight": 1
> }]
and the logstash conf like this:
> input {
> file {
> path =\> "/somepath/waf/\*.log"
> codec =\> "json"
> discover\_interval =\> 2
> }
> }
> filter {
> json {
> source =\> "message"
> }
> date {
> match =\> ["[Timestamp]", "ISO8601"]
> target =\> "@timestamp"
> remove\_field =\> "timestamp"
> }
> split {
> field =\> "[Request][Headers]"
> }
> }
> output {
> elasticsearch {
> hosts =\> "[https://someelk:443](https://someelk:443)"
> index =\> "waf-%{+YYYY.MM.dd}"
> ssl\_certificate\_verification =\> "false"
> }
> }
Everything works except because Json array Name and Value are added as fields, but what i need is the **"Name" key value added as fields** and the **"Value" key value xD as the Name field values**
Ex of result wanted: **a new field called Request.Header.Host and its value to be [www.example.net](http://www.example.net)**
Any hints? Sorry that might be an easy one....
Thanks!
---
### Author:  [@darkmoon](https://discuss.elastic.co/u/darkmoon)
#### Post date: [May 3, 2018, 3:08pm UTC](https://discuss.elastic.co/t/solved-split-filter-question-a-k-a-flatten-json-sub-array/130481/2 "2018-05-03T15:08:29Z")
Maybe a mutate? Add the value of Value to a filed with a name of Name? You run the risk of generating an arbitrary number of fields then.
---
### Author:  [@blastik](https://discuss.elastic.co/u/blastik)
#### Post date: [May 3, 2018, 3:09pm UTC](https://discuss.elastic.co/t/solved-split-filter-question-a-k-a-flatten-json-sub-array/130481/3 "2018-05-03T15:09:50Z")
would you put an example?
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 3, 2018, 3:19pm UTC](https://discuss.elastic.co/t/solved-split-filter-question-a-k-a-flatten-json-sub-array/130481/4 "2018-05-03T15:19:32Z")
> [@blastik](#):
>
> remove\_field =\> "timestamp"
Field names are case sensitive, your field is named Timestamp, not timestamp.
```
mutate { add_field => { "%{[Request][Headers][Name]}" => "%{[Request][Headers][Value]}" } }
```
Will do it if I understood the ask correctly.
---
### Author:  [@blastik](https://discuss.elastic.co/u/blastik)
#### Post date: [May 3, 2018, 4:53pm UTC](https://discuss.elastic.co/t/solved-split-filter-question-a-k-a-flatten-json-sub-array/130481/5 "2018-05-03T16:53:49Z")
sorry but no luck. its adding those fields but nothing else.
btw, thanks for the heads up in the timestamp typo
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 3, 2018, 5:06pm UTC](https://discuss.elastic.co/t/solved-split-filter-question-a-k-a-flatten-json-sub-array/130481/6 "2018-05-03T17:06:48Z")
Having re-read what you asked for, does this work?
```
mutate { add_field => { "[Request][Headers]%{[Request][Headers][Name]}" => "%{[Request][Headers][Value]}" } }
```
That gives you
```auto
"Request" => {
"ClientIP" => "8.8.8.8",
"HTTPVersion" => "HTTP/2.0",
"Country" => "ES",
"URI" => "/uploadMultiplePhotos.aspx",
"Headers" => {
"Host" => "www.example.net",
"Name" => "Host",
"Value" => "www.example.net"
},
"Method" => "POST"
},
```
Why do you split on [Request][Headers]?
---
### Author:  [@blastik](https://discuss.elastic.co/u/blastik)
#### Post date: [May 3, 2018, 5:34pm UTC](https://discuss.elastic.co/t/solved-split-filter-question-a-k-a-flatten-json-sub-array/130481/7 "2018-05-03T17:34:36Z")
sorry to be a pain but still no luck.
here is how it looks the filter:
> filter {
> json {
> source =\> "message"
> }
> date {
> match =\> ["[Timestamp]", "ISO8601"]
> target =\> "@timestamp"
> remove\_field =\> "Timestamp"
> }
> mutate {
> add\_field =\> {
> "[Request][Headers]%{[Request][Headers][Name]}" =\> "%{[Request][Headers][Value]}"
> }
> }
> }
i splitted request headers to test. but i didnt work
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 3, 2018, 5:44pm UTC](https://discuss.elastic.co/t/solved-split-filter-question-a-k-a-flatten-json-sub-array/130481/8 "2018-05-03T17:44:28Z")
So what does output { stdout { codec =\> rubydebug } } produce, and what don't you like about it?
BTW, if you have a json codec on the input, the json filter is not needed.
---
### Author:  [@blastik](https://discuss.elastic.co/u/blastik)
#### Post date: [May 3, 2018, 5:50pm UTC](https://discuss.elastic.co/t/solved-split-filter-question-a-k-a-flatten-json-sub-array/130481/9 "2018-05-03T17:50:12Z")
thank you, i've **removed** the below now
> json {
> source =\> "message"
> }
it makes no difference. it creates a single field
> Request.Headers
with all the data inside ☹
---
### Author:  [@blastik](https://discuss.elastic.co/u/blastik)
#### Post date: [May 3, 2018, 6:00pm UTC](https://discuss.elastic.co/t/solved-split-filter-question-a-k-a-flatten-json-sub-array/130481/10 "2018-05-03T18:00:46Z")
let me add that the json looks more like this, so it contains several objects
> [{
> "Action": "COUNT",
> "Timestamp": "2018-05-02T15:36:14Z",
> "Request": {
> "Country": "PT",
> "URI": "/uploadMultiplePhotos.aspx",
> "Headers": [{
> "Name": "Host",
> "Value": "[www.example.net](http://www.example.net)"
> }, {
> "Name": "Content-Length",
> "Value": "175253"
> }, {
> "Name": "origin",
> "Value": "[https://www.example.net](https://www.example.net)"
> }, {
> "Name": "user-agent",
> "Value": "Mozilla/5.0 (Linux; Android 7.0; LG-M200 Build/NRD90U) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.126 Mobile Safari/537.36"
> }, {
> "Name": "content-type",
> "Value": "multipart/form-data; boundary=----WebKitFormBoundaryXBuUMINHBoY9jHwN"
> }, {
> "Name": "accept",
> "Value": "application/json"
> }, {
> "Name": "cache-control",
> "Value": "no-cache"
> }, {
> "Name": "x-requested-with",
> "Value": "XMLHttpRequest"
> }, {
> "Name": "save-data",
> "Value": "on"
> }, {
> "Name": "x-ajax",
> "Value": "blabla"
> }, {
> "Name": "referer",
> "Value": "[https://www.example.net/blabla](https://www.example.net/blabla)"
> }, {
> "Name": "accept-encoding",
> "Value": "gzip, deflate, br"
> }, {
> "Name": "accept-language",
> "Value": "pt-PT,pt;q=0.9,en-US;q=0.8,en;q=0.7,es;q=0.6"
> }, {
> "Name": "cookie",
> "Value": "blabla"
> }],
> "ClientIP": "8.8.8.8",
> "Method": "POST",
> "HTTPVersion": "HTTP/2.0"
> },
> "Weight": 1
> },{
> "Action": "COUNT",
> "Timestamp": "2018-05-02T15:36:14Z",
> "Request": {
> "Country": "PT",
> "URI": "/uploadMultiplePhotos.aspx",
> "Headers": [{
> "Name": "Host",
> "Value": "[www.example.net](http://www.example.net)"
> }, {
> "Name": "Content-Length",
> "Value": "175253"
> }, {
> "Name": "origin",
> "Value": "[https://www.example.net](https://www.example.net)"
> }, {
> "Name": "user-agent",
> "Value": "Mozilla/5.0 (Linux; Android 7.0; LG-M200 Build/NRD90U) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.126 Mobile Safari/537.36"
> }, {
> "Name": "content-type",
> "Value": "multipart/form-data; boundary=----WebKitFormBoundaryXBuUMINHBoY9jHwN"
> }, {
> "Name": "accept",
> "Value": "application/json"
> }, {
> "Name": "cache-control",
> "Value": "no-cache"
> }, {
> "Name": "x-requested-with",
> "Value": "XMLHttpRequest"
> }, {
> "Name": "save-data",
> "Value": "on"
> }, {
> "Name": "x-ajax",
> "Value": "blabla"
> }, {
> "Name": "referer",
> "Value": "[https://www.example.net/blabla](https://www.example.net/blabla)"
> }, {
> "Name": "accept-encoding",
> "Value": "gzip, deflate, br"
> }, {
> "Name": "accept-language",
> "Value": "pt-PT,pt;q=0.9,en-US;q=0.8,en;q=0.7,es;q=0.6"
> }, {
> "Name": "cookie",
> "Value": "blabla"
> }],
> "ClientIP": "8.8.8.8",
> "Method": "POST",
> "HTTPVersion": "HTTP/2.0"
> },
> "Weight": 1
> },{
> "Action": "COUNT",
> "Timestamp": "2018-05-02T15:36:14Z",
> "Request": {
> "Country": "PT",
> "URI": "/uploadMultiplePhotos.aspx",
> "Headers": [{
> "Name": "Host",
> "Value": "[www.example.net](http://www.example.net)"
> }, {
> "Name": "Content-Length",
> "Value": "175253"
> }, {
> "Name": "origin",
> "Value": "[https://www.example.net](https://www.example.net)"
> }, {
> "Name": "user-agent",
> "Value": "Mozilla/5.0 (Linux; Android 7.0; LG-M200 Build/NRD90U) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.126 Mobile Safari/537.36"
> }, {
> "Name": "content-type",
> "Value": "multipart/form-data; boundary=----WebKitFormBoundaryXBuUMINHBoY9jHwN"
> }, {
> "Name": "accept",
> "Value": "application/json"
> }, {
> "Name": "cache-control",
> "Value": "no-cache"
> }, {
> "Name": "x-requested-with",
> "Value": "XMLHttpRequest"
> }, {
> "Name": "save-data",
> "Value": "on"
> }, {
> "Name": "x-ajax",
> "Value": "blabla"
> }, {
> "Name": "referer",
> "Value": "[https://www.example.net/blabla](https://www.example.net/blabla)"
> }, {
> "Name": "accept-encoding",
> "Value": "gzip, deflate, br"
> }, {
> "Name": "accept-language",
> "Value": "pt-PT,pt;q=0.9,en-US;q=0.8,en;q=0.7,es;q=0.6"
> }, {
> "Name": "cookie",
> "Value": "blabla"
> }],
> "ClientIP": "8.8.8.8",
> "Method": "POST",
> "HTTPVersion": "HTTP/2.0"
> },
> "Weight": 1
> }]
---
### Author:  [@blastik](https://discuss.elastic.co/u/blastik)
#### Post date: [May 3, 2018, 6:15pm UTC](https://discuss.elastic.co/t/solved-split-filter-question-a-k-a-flatten-json-sub-array/130481/11 "2018-05-03T18:15:06Z")
mmmm just payed attention on the `[` and `]` at the beginning and at the end of the json... 🤔
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 3, 2018, 6:17pm UTC](https://discuss.elastic.co/t/solved-split-filter-question-a-k-a-flatten-json-sub-array/130481/12 "2018-05-03T18:17:09Z")
So what does output { stdout { codec =\> rubydebug } } produce, and what don't you like about it?
My guess is you will end up wanting something like
```auto
ruby {
code => '
event.get("[Request][Headers]").each { |a|
name = a["Name"]
value = a["Value"]
event.set( "[Request][HeadersFlattened]#{name}", value)
}
'
}
```
Which will get you this if your event is a single JSON object from that array.
```auto
"HeadersFlattened" => {
"accept" => "application/json",
"content-type" => "multipart/form-data; boundary=----WebKitFormBoundaryBnMGtRJgyfhSDZt3",
"cookie" => "GREEDY",
"origin" => "https://www.example.net",
"user-agent" => "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36",
"cache-control" => "no-cache",
"x-requested-with" => "XMLHttpRequest",
"Host" => "www.example.net",
"referer" => "https://www.example.net/bla.aspx",
"Content-Length" => "226245",
"accept-encoding" => "gzip, deflate, br",
"accept-language" => "es-ES,es;q=0.8",
"x-ajax" => "example"
},
```
---
### Author:  [@blastik](https://discuss.elastic.co/u/blastik)
#### Post date: [May 3, 2018, 6:31pm UTC](https://discuss.elastic.co/t/solved-split-filter-question-a-k-a-flatten-json-sub-array/130481/13 "2018-05-03T18:31:23Z")
yep that made the trick 🤪😃. however just for the first of the objects.... then the rest look like this:
> "Headers" =\> [
> [0] {
> "Name" =\> "Host",
> "Value" =\> "[www.example.net](http://www.example.net)"
> },
> [1] {
> "Name" =\> "Content-Length",
> "Value" =\> "90726"
> },
> [2] {
> "Name" =\> "origin",
> "Value" =\> "[https://www.example.net](https://www.example.net)"
> },
> [3] {
> "Name" =\> "user-agent",
> "Value" =\> "Mozilla/5.0 (Linux; Android 7.0; RNE-L21 Build/HUAWEIRNE-L21) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.126 Mobile Safari/537.36"
> },
> [4] {
> "Name" =\> "content-type",
> "Value" =\> "multipart/form-data; boundary=----WebKitFormBoundaryU0dmYmrvc8iBUZ8C"
> },
> [5] {
> "Name" =\> "accept",
> "Value" =\> "application/json"
> },
> [6] {
> "Name" =\> "cache-control",
> "Value" =\> "no-cache"
> },
> [7] {
> "Name" =\> "x-requested-with",
> "Value" =\> "XMLHttpRequest"
> },
> [8] {
> "Name" =\> "x-ajax",
> "Value" =\> "blabla"
> },
> [9] {
> "Name" =\> "referer",
> "Value" =\> "[https://www.example.net/blae.aspx](https://www.example.net/blae.aspx)"
> },
> [10] {
> "Name" =\> "accept-encoding",
> "Value" =\> "gzip, deflate, br"
> },
> [11] {
> "Name" =\> "accept-language",
> "Value" =\> "es-ES,es;q=0.9"
> },
> [12] {
> "Name" =\> "cookie",
> "Value" =\> "blabla"
> }
> ]
---
### Author:  [@blastik](https://discuss.elastic.co/u/blastik)
#### Post date: [May 3, 2018, 7:11pm UTC](https://discuss.elastic.co/t/solved-split-filter-question-a-k-a-flatten-json-sub-array/130481/14 "2018-05-03T19:11:35Z")
oh wait i think because i need to remove "[Request][Headers]" after being processed by ruby block.... let me try!
---
### Author:  [@blastik](https://discuss.elastic.co/u/blastik)
#### Post date: [May 3, 2018, 7:14pm UTC](https://discuss.elastic.co/t/solved-split-filter-question-a-k-a-flatten-json-sub-array/130481/15 "2018-05-03T19:14:54Z")
yes!!!! thank you badger you rock!
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 31, 2018, 7:14pm UTC](https://discuss.elastic.co/t/solved-split-filter-question-a-k-a-flatten-json-sub-array/130481/16 "2018-05-31T19:14:56Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.