# Logstash kv filter extract json array issue
**URL:** https://discuss.elastic.co/t/logstash-kv-filter-extract-json-array-issue/163835
**Category:** Logstash
**Created:** [January 11, 2019, 2:45am UTC](https://discuss.elastic.co/t/logstash-kv-filter-extract-json-array-issue/163835 "2019-01-11T02:45:33Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)
#### Post date: [January 11, 2019, 2:45am UTC](https://discuss.elastic.co/t/logstash-kv-filter-extract-json-array-issue/163835/1 "2019-01-11T02:45:33Z")
Hi,
**I have below json file, and the json object contains array. I want to use kv filter ( not json filter) to extract corresponding kv paris**,
{
"DP": "dbpool1",
"CMID": "TALSCMK",
"CMN": "HuaWei",
"UID": "lokamoto1",
"UN": "lokamoto",
"PUID": "admin1",
"UL": "zh\_CN",
"CIP": "192.168.56.3",
"SN": "qacandrot\_TALSCMK",
"DC": "DC08",
"CLN": "TalentSearchController",
"MID": "SCM",
"PID": "TalentSearch",
"PQ": "v11",
"AC": "TalentSearch",
"SCM.TS.TS.IIN": "true",
"SCM.TS.TS.MACO": "false",
**"SCM.TS.TS.COND": ["KC", "BC", "PC", "FC", "RC"],**
\*\* "SCM.TS.TS.BC": ["age", "fax", "ethnicity"],\*\*
\*\* "SCM.TS.TS.PC": ["achievements", "languages"],\*\*
\*\* "SCM.TS.TS.FC": ["department", "location"],\*\*
\*\* "SCM.TS.TS.RC": ["sysOverallPotential", "sysOverallCustom1"],\*\*
"SCM.TS.TS.NR": 200
}
**My kv pair configured**
kv {
source =\> "message"
allow\_duplicate\_values =\> true
field\_split =\> ","
value\_split =\> ":"
```
}
```
extracted information after kv filter
""CMN"" =\> "HuaWei",
""UN"" =\> "lokamoto",
""PUID"" =\> "admin1",
""AC"" =\> "TalentSearch",
""SCM.TS.TS.IIN"" =\> "true",
""SCM.TS.TS.RC"" =\> ""sysOverallPotential","sysOverallCustom1"",
""SCM.TS.TS.PC"" =\> ""achievements","languages"",
""CMID"" =\> "TALSCMK",
""SCM.TS.TS.BC"" =\> ""age","fax","ethnicity"",
""SN"" =\> "qacandrot\_TALSCMK",
""SCM.TS.TS.COND"" =\> ""KC","BC","PC","FC","RC"",
""PID"" =\> "TalentSearch",
"@version" =\> "1",
"host" =\> "PVGN50859047A",
""SCM.TS.TS.NR"" =\> "200}\r",
""SCM.TS.TS.FC"" =\> ""department","location"",
""PQ"" =\> "v11",
""DC"" =\> "DC08",
"path" =\> "C:\elkstack\elasticsearch-6.5.1\logs\kv.log",
""MID"" =\> "SCM",
""UL"" =\> "zh\_CN",
""CIP"" =\> "192.168.56.3",
"{"DP"" =\> "dbpool1",
""CLN"" =\> "TalentSearchController",
"@timestamp" =\> 2019-01-11T02:50:04.029Z,
""SCM.TS.TS.MACO"" =\> "false",
""UID"" =\> "lokamoto1"
**In Kibana, the display looks like below after kv filter, and when doing aggregation based on key whose value is array. For example, CM.TS.TS.COND, kibana treats corresponding value "KC", "BC", "PC", "FC", "RC" as string, instead of array.**
**Expected Chart I want get is to aggregate based on the values in the array.**
**How should I configure kv filter to let kibana aggreate the chart based on arrays?**
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [January 11, 2019, 3:49am UTC](https://discuss.elastic.co/t/logstash-kv-filter-extract-json-array-issue/163835/2 "2019-01-11T03:49:37Z")
This is what the JSON filter was designed for, why not use it?
---
### Author:  [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)
#### Post date: [January 12, 2019, 12:28am UTC](https://discuss.elastic.co/t/logstash-kv-filter-extract-json-array-issue/163835/3 "2019-01-12T00:28:58Z")
I am just seeking for a kv filter solution. (json filter does work)
---
### Author:  [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)
#### Post date: [January 12, 2019, 2:21am UTC](https://discuss.elastic.co/t/logstash-kv-filter-extract-json-array-issue/163835/4 "2019-01-12T02:21:00Z")
Any idea how to make it by kv filter or a combination filters?
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [January 12, 2019, 7:45am UTC](https://discuss.elastic.co/t/logstash-kv-filter-extract-json-array-issue/163835/5 "2019-01-12T07:45:41Z")
No, I do not know. As there is a filter specifically designed for parsing this I also do not see the point in this exercise, so will unfortunately not be able to help.
---
### Author:  [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)
#### Post date: [January 16, 2019, 8:29am UTC](https://discuss.elastic.co/t/logstash-kv-filter-extract-json-array-issue/163835/6 "2019-01-16T08:29:05Z")
Hi Christian,
I think I found the solution. Configure below filters with KV+Ruby returns my expected behavior in Kibana. the pie chart can be aggrated based on array elements successfully.
```
kv {
source => "message"
field_split => ","
value_split => ":"
}
ruby {
code => "
event.to_hash.each { |k,v|
if v.is_a?(String)&&v.length>2
event.set(k, v.split(','))
else
event.set(k, v)
end
}"
}
```
---
### Author:  [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)
#### Post date: [January 16, 2019, 8:33am UTC](https://discuss.elastic.co/t/logstash-kv-filter-extract-json-array-issue/163835/7 "2019-01-16T08:33:05Z")
AND more easiler way is the json filter, either specify it in input, filter or output.
1. configure in input
input{
file{
path =\> "C:\elkstack\elasticsearch-6.5.1\logs\kv.log"
start\_position =\> "beginning"
sincedb\_path =\> "null"
codec =\> "json"
}
}
1. configure in filter
filter {
json {
source =\> "message"
}
}
2. configure in output
stdout {
codec =\> "json"
}
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 13, 2019, 8:33am UTC](https://discuss.elastic.co/t/logstash-kv-filter-extract-json-array-issue/163835/8 "2019-02-13T08:33:12Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.