# Keyword aggregation returns empty
**URL:** https://discuss.elastic.co/t/keyword-aggregation-returns-empty/110480
**Category:** Elasticsearch
**Created:** [December 6, 2017, 9:34am UTC](https://discuss.elastic.co/t/keyword-aggregation-returns-empty/110480 "2017-12-06T09:34:59Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@yishain11](https://discuss.elastic.co/u/yishain11)
#### Post date: [December 6, 2017, 9:35am UTC](https://discuss.elastic.co/t/keyword-aggregation-returns-empty/110480/1 "2017-12-06T09:35:00Z")
Hi everyone!
I have a problem with elasticsearch. I get data via filebeat and logstash to kibana.
I add mapping as keyword to certien words in the data, and even though I see thous keyword under "message" field in kibana, the keywords aggregations themselves return empty queries.
sample data:
```
{
"DataChangeInfo": "Archive Log Set archiveLogSet.108058603.108058399 Info:Thread# Sequence#FirstScn LastScn ",
"documentsList": [{
"commandScn": "108058599",
"commandCommitScn": "108058600",
"commandSequence": "0",
"commandType": "UPDATE",
"commandTimestamp": "2017-08-22 14:37:53+03:000",
"objectDBName": "DEV2",
"objectSchemaName": "YISHAIN",
"objectId": "CUSTOMERS",
"changedFieldsList": [{
"fieldId": "CUSTOMER_ID",
"fieldType": "NUMBER",
"fieldValue": "17",
"fieldChanged": "N"
}, {
"fieldId": "CUSTOMER_FIRST_NAME",
"fieldType": "VARCHAR2",
"fieldValue": "Daniel",
"fieldChanged": "N"
}, {
"fieldId": "CUSTOMER_LAST_NAME",
"fieldType": "VARCHAR2",
"fieldValue": "Washington",
"fieldChanged": "N"
}, {
"fieldId": "LAST_UPDTAE",
"fieldType": "DATE",
"fieldValue": "2016-01-12 00:00:00+03:000",
"fieldChanged": "N"
}],
"conditionFieldsList": [{
"fieldId": "CUSTOMER_ID",
"fieldType": "NUMBER",
"fieldValue": "17"
}, {
"fieldId": "CUSTOMER_FIRST_NAME",
"fieldType": "VARCHAR2",
"fieldValue": "Daniel"
}, {
"fieldId": "LAST_UPDTAE",
"fieldType": "DATE",
"fieldValue": "2016-01-12 00:00:00+03:000"
}]
}]
}
```
the relevant mapping:
documentsList: nested.
changedFieldsList: nested.
fieldId, fieldType, fieldValue, fieldChanged: text + keyword.
```
{
"filebeat-2017.12.05": {
"mappings": {
"_default_": {
"_meta": {
"version": "5.6.4"
},
"dynamic_templates": [{
"strings_as_keyword": {
"match_mapping_type": "string",
"mapping": {
"ignore_above": 1024,
"type": "keyword"
}
}
}],
"date_detection": false,
"properties": {}
},
"log": {
"dynamic_templates": [{
"strings_as_keyword": {
"match_mapping_type": "string",
"mapping": {
"ignore_above": 1024,
"type": "keyword"
}
}
}],
"date_detection": false,
"properties": {
"DataChangeInfo": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
}
},
"changedFieldList": {
"properties": {
"fieldChanged": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
}
},
"fieldId": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
}
},
"fieldType": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
}
},
"fieldValue": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
}
}
}
},
"changedFieldsList": {
"type": "nested"
},
"documentsList": {
"type": "nested"
},
"fieldChanged": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
}
},
"fieldId": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
}
},
"fieldType": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
}
},
"fieldValue": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
}
}
}
}
}
}
}
```
filebeat config:
```
filebeat.prospectors:
- input_type: log
paths:
- /path/to/file/*
output.logstash:
hosts: [myIP]
```
logstash config:
```
input {
beats {
port => 5044
}
}
filter{
}
output {
elasticsearch {
hosts => ["http://myIP"]
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
```
example to empty search under the "fileType" keyword, even though it does appear under "message".
It happen in all keywords I defined in the mapping.
Why is this happening? How can I aggregate by the terms I defined as keyword?
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [December 6, 2017, 10:03am UTC](https://discuss.elastic.co/t/keyword-aggregation-returns-empty/110480/2 "2017-12-06T10:03:07Z")
Not sure but I see a "changedFieldsList" plural in the example doc but a "changedFieldList", singular in the mapping.
---
### Author:  [@yishain11](https://discuss.elastic.co/u/yishain11)
#### Post date: [December 6, 2017, 10:31am UTC](https://discuss.elastic.co/t/keyword-aggregation-returns-empty/110480/3 "2017-12-06T10:31:54Z")
Thanks for the fix. I changed the mapping.
The problem still presists, though.
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [December 6, 2017, 10:56am UTC](https://discuss.elastic.co/t/keyword-aggregation-returns-empty/110480/4 "2017-12-06T10:56:36Z")
The "message" field from your Kibana screenshot looks pretty messed up.
Looks like some original JSON strings that have been split into multiple documents on carriage return boundaries. Check the raw JSON you are sending to elasticsearch
---
### Author:  [@yishain11](https://discuss.elastic.co/u/yishain11)
#### Post date: [December 6, 2017, 12:52pm UTC](https://discuss.elastic.co/t/keyword-aggregation-returns-empty/110480/5 "2017-12-06T12:52:22Z")
The files are fine.
I tried to use the JSON filter in logstash, but I removed the filter, as you can see.
Maybe its still work? I start logstash as service in CentOS 7, but there is only 1 config file in conf.d, and it doesn't have the filter anymore.
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [December 6, 2017, 1:16pm UTC](https://discuss.elastic.co/t/keyword-aggregation-returns-empty/110480/6 "2017-12-06T13:16:44Z")
> [@yishain11](#):
>
> The files are fine.
The files may be fine but what Logstash sends to elasticsearch might be less so.
Let's start by looking at the raw JSON of one of the docs in your Kibana screenshot (the clickable arrow next to each doc should help reveal it.) If that doesn't look like what you expect then the question is really for the logstash forum.
---
### Author:  [@yishain11](https://discuss.elastic.co/u/yishain11)
#### Post date: [December 6, 2017, 1:22pm UTC](https://discuss.elastic.co/t/keyword-aggregation-returns-empty/110480/7 "2017-12-06T13:22:32Z")
here it is:
it looks fine other than the fields I want to aggregate by don't appear.
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [December 6, 2017, 1:25pm UTC](https://discuss.elastic.co/t/keyword-aggregation-returns-empty/110480/8 "2017-12-06T13:25:51Z")
Can you click the JSON tab just to confirm the raw data you have presented to elasticsearch?
Elasticsearch can't aggregate what isn't there so if the JSON is not right you need to be asking in the logstash forum about how it has transformed your source data.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 3, 2018, 1:26pm UTC](https://discuss.elastic.co/t/keyword-aggregation-returns-empty/110480/9 "2018-01-03T13:26:27Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.