# Aggregate by keyword in kibana 5.6 **URL:** \ **Category:** Kibana\ **Created:** [November 27, 2017, 9:41am UTC](https://discuss.elastic.co/t/aggregate-by-keyword-in-kibana-5-6/109193 "2017-11-27T09:41:12Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![yishain11](https://avatars.discourse-cdn.com/v4/letter/y/df705f/32.png) [@yishain11](https://discuss.elastic.co/u/yishain11)\ **Post date:** [November 27, 2017, 9:41am UTC](https://discuss.elastic.co/t/aggregate-by-keyword-in-kibana-5-6/109193/1 "2017-11-27T09:41:12Z")
Hi everyone! I have the following setup: filebeat (collects log file) \> logdash (filters JSON) \> elasticsearch \> kibana. In kibana, I mapped the data as follows: get filebeat-2017\*/\_mapping/log results in: ``` . . . "commandCommitScn": { "type": "long", "fields": { "raw": { "type": "keyword" } } }, "commandScn": { "type": "long", "fields": { "raw": { "type": "keyword" } } }, "commandSequence": { "type": "long", "fields": { "raw": { "type": "keyword" } } }, "commandTimestamp": { "type": "date", "fields": { "raw": { "type": "keyword" } } }, "commandType": { "type": "keyword", "fields": { "raw": { "type": "keyword" } } }, "error": { "type": "keyword", "ignore_above": 1024 }, "fieldChanged": { "type": "keyword", "fields": { "raw": { "type": "keyword" } } }, "fieldId": { "type": "keyword", "fields": { "raw": { "type": "keyword" } } }, "fieldType": { "type": "keyword", "fields": { "raw": { "type": "keyword" } } }, "fieldValue": { "type": "keyword", "fields": { "raw": { "type": "keyword" } } }, "fileset": { "properties": { "module": { "type": "keyword", "ignore_above": 1024 }, "name": { "type": "keyword", "ignore_above": 1024 } } ... ``` as you can see, the terms I want to aggragate by are defined as keywords in the mapping. When I use Discover in kibana, the message field is: ``` ... "fieldType": "VARCHAR2", "fieldValue": "85928 CEDEX 9" "fieldType": "DATE", ... ``` So it seems like kibana understands that this terms are not plain text, but a terms that need to be read by themselves. The problem is that I can't find an option in the kibana aggregation in Visualize, that will anable me to aggregate by this fields (more specificly - by there results, for example: how many "fieldType" are dates and how many are other types). How can I aggregate by the keywords I want? This is the logstash confic I use: ``` input { beats { port => 5044 } } filter{ split{ field => "changedFieldList" } json{ source => "message" target => "parsedJson" } } output { elasticsearch { hosts => ["http://127.0.0.1:9200"] index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}" document_type => "%{[@metadata][type]}" } } ``` And the file I insert is something like this: ``` { "ChangeInfo": "Archive Log Set archiveLogSet.108058603.108058399 Info:Thread# Sequence# FirstScn LastScn ", "documentsList": [{ "commandScn": "108058599", "commandCommitScn": "108058600", "commandSequence": "0", "commandType": "UPDATE", "commandTimestamp": "2017-08-22 14:37:53+03:000", "objectDBName": "DEV2", "objectSchemaName": "YISHAIN", "objectId": "CUSTOMERS", "changedFieldsList": [{ "fieldId": "CUSTOMER_ID", "fieldType": "NUMBER", "fieldValue": "17", "fieldChanged": "N" }, { "fieldId": "GENDER", "fieldType": "VARCHAR2", "fieldValue": "Male", "fieldChanged": "N" }, { "fieldId": "DATE_OF_BIRTH", "fieldType": "DATE", "fieldValue": "1968-04-01 00:00:00+03:000", "fieldChanged": "N" }, { "fieldId": "CREDIT_LIMIT", "fieldType": "NUMBER", "fieldValue": "14161", "fieldChanged": "N" }, { "fieldId": "INCOME_LEVEL", "fieldType": "NUMBER", "fieldValue": "67035", "fieldChanged": "N" }, { "fieldId": "COMMENTS", "fieldType": "VARCHAR2", "fieldValue": "stuff", "fieldChanged": "N" }, { "fieldId": "LAST_UPDTAE", "fieldType": "DATE", "fieldValue": "2016-01-12 00:00:00+03:000", "fieldChanged": "N" }], "conditionFieldsList": [{ "fieldId": "CUSTOMER_ID", "fieldType": "NUMBER", "fieldValue": "17" }, { "fieldId": "CUSTOMER_FIRST_NAME", "fieldType": "VARCHAR2", "fieldValue": "Daniel" }, { "fieldId": "CUSTOMER_LAST_NAME", "fieldType": "VARCHAR2", "fieldValue": "Washington" }, { "fieldId": "CUSTOMER_COUNTRY", "fieldType": "VARCHAR2", "fieldValue": "France" }, { "fieldId": "CUSTOMER_CITY", "fieldType": "VARCHAR2", "fieldValue": "La Roche-sur-Yon" }, { "fieldId": "CUSTOMER_STREET", "fieldType": "VARCHAR2", "fieldValue": "5 Maple Pass" }, { "fieldId": "CUSTOMER_ZIPCODE", "fieldType": "VARCHAR2", "fieldValue": "85928 CEDEX 9" }, { "fieldId": "PHONE_NUMBER", "fieldType": "VARCHAR2", "fieldValue": "33-(414)618-3273" }, { "fieldId": "EMAIL", "fieldType": "VARCHAR2", "fieldValue": "dwashingtong@tiny.cc" }, { "fieldId": "GENDER", "fieldType": "VARCHAR2", "fieldValue": "Male" }, { "fieldId": "DATE_OF_BIRTH", "fieldType": "DATE", "fieldValue": "1968-04-01 00:00:00+03:000" }, { "fieldId": "CREDIT_LIMIT", "fieldType": "NUMBER", "fieldValue": "14161" }, { "fieldId": "INCOME_LEVEL", "fieldType": "NUMBER", "fieldValue": "67035" }, { "fieldId": "COMMENTS", "fieldType": "VARCHAR2", "fieldValue": "stuff" }, { "fieldId": "LAST_UPDTAE", "fieldType": "DATE", "fieldValue": "2016-01-12 00:00:00+03:000" }] }] } ``` What do I need to do in order to aggregate and visualize by the terms in the JSON file? Thanks! ---
**Author:** ![tylersmalley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tylersmalley/32/8833_2.png) [@tylersmalley](https://discuss.elastic.co/u/tylersmalley)\ **Post date:** [November 28, 2017, 5:56am UTC](https://discuss.elastic.co/t/aggregate-by-keyword-in-kibana-5-6/109193/2 "2017-11-28T05:56:41Z")
In Kibana under your index pattern in Management do you see that the fields are aggregatable? What if you click the refresh field list button? ---
**Author:** ![yishain11](https://avatars.discourse-cdn.com/v4/letter/y/df705f/32.png) [@yishain11](https://discuss.elastic.co/u/yishain11)\ **Post date:** [November 28, 2017, 7:39am UTC](https://discuss.elastic.co/t/aggregate-by-keyword-in-kibana-5-6/109193/3 "2017-11-28T07:39:59Z")
After I refreshed the index pattern like you ask, I see my field in there, and I see that they are aggregateable. The problem is that in Discover I still can't aggregate by them, even when I search for them, and in Visualize they don't apear ether. ![kibana](https://us1.discourse-cdn.com/elastic/original/3X/7/9/795cd37347a4ee8cc8466f710f627594309f1325.JPG) EDIT: I was able to find these fields in the Discover page, but they seem to be empty, even if they do appear under the "message" field. Why is this happening? ![kibana2](https://us1.discourse-cdn.com/elastic/original/3X/b/b/bbb29b30c7a6605cc917b3bab8fb95b439fdc42a.JPG) ---
**Author:** ![yishain11](https://avatars.discourse-cdn.com/v4/letter/y/df705f/32.png) [@yishain11](https://discuss.elastic.co/u/yishain11)\ **Post date:** [December 5, 2017, 1:15pm UTC](https://discuss.elastic.co/t/aggregate-by-keyword-in-kibana-5-6/109193/4 "2017-12-05T13:15:02Z")
anyone? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 2, 2018, 1:15pm UTC](https://discuss.elastic.co/t/aggregate-by-keyword-in-kibana-5-6/109193/5 "2018-01-02T13:15:03Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.