[Resolved][kinaba7.0]Terms aggregration doesn;t aggregrate by individual values in array

[cheriemilk]

Hi team,
I am running into an issue in ELK production environment and can’t figure out why googled. The google result is saying that it might because the ktf1.raw field not enabled. But I still failed after apply the raw field. Can you help have a look and advice?

Issue Description:

1. I have below data in elk, the value for ktf1, kf1, kf2, ktf3 are arrays. In Kibana, I want the it aggregate by the values in the array. But I didn’t get the expected result. It treat the array as a string, and aggregrate based on the string. Please refer to the screenshot in point3 .

at,ktf1,kf1,kf2,kf3
SAVE,"[performance,compensation]","[true,true,false,false]","[false,false,false,false]","[false,false,false,false]"
SAVE,"[performance,compensation,Others]","[true,true,true,false]","[false,false,false,false]","[true,true,false,false]"
SAVE,"[performance,compensation,Others]","[true,true,true,false]","[false,false,false,false]","[true,true,false,false]"
SAVE,"[performance,compensation,Others]","[true,true,true,false]","[false,false,false,false]","[true,true,false,false]"
SAVE,"[performance,compensation,liveProfile,talentFlag,Others]","[true,false,false,false]","[false,false,false,false]","[true,false,false,false]"

2. The index template definition for ktf1, kf1, kf2, ktf3 are below.
   > {

          "template": "kvaudit",
          "index_patterns": ["kvaudit"],
          "settings": {
          "index": {
               "number_of_shards": "1",
               "codec": "best_compression",
               "number_of_replicas": "0"
                  }
              },
        "mappings": {
           "doc": {
           "properties": {
           "@version": {
           "type": "keyword" }
        }
           }
             },
  "beat": { 
        	"properties": {
        	"version": {
        "type": "keyword"
            }
              }
                },

"fields": {
"properties": { "at": {"type": "keyword"},
	            "ktf1": {"type": "text","fields": {"raw": {"type": "keyword","ignore_above": 256}}},
                "ktf2": {"type": "keyword"},
                "ktf3": {"type": "keyword"},
                "ktf4": {"type": "keyword"},
                "ktf5": {"type": "keyword"},
                "kf1": {"type": "keyword"},
                "kf2": {"type"："keyword"},
                "kf3": {"type"："keyword"}}}
}

3. Screenshot of Terms aggregration
   
   

[Marius_Dragomir]

Kibana doesn't fully support nested objects and arrays so this might be why it's not working for you. But, a quick question first: after you made the field analyzed, did you reindex the data so that it got analyzed by Elasticsearch?

[cheriemilk]

Hi Marius,
I have resolved it by adding a ruby filter to split string by ',' in logstash.conf.

Thanks,
Cherie