# Is there any length limitation of the key in term aggregation ? I mean results display
**URL:** https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024
**Category:** Elasticsearch
**Created:** [August 15, 2016, 11:55am UTC](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024 "2016-08-15T11:55:05Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@pill663](https://discuss.elastic.co/u/pill663)
#### Post date: [August 15, 2016, 11:55am UTC](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024/1 "2016-08-15T11:55:05Z")
I have some long string documents stored in elasticsearch(version 2.3).
I want to do a term aggregation for my documents. I choose one Field for the term , the records that stored in this field is some long strings.
the total count of the docs is 718. But the aggregation result is much less than the total count.
I think some long docs are filtered or dropped because the string is too long, some are longer than 256 characters.
So, I want to ask 'Is there any length limitation of the key in term aggregation'?
Is there any way to break this limitation?
You can see these are the aggregation results:
{
"took": 100,
"timed\_out": false,
"\_shards": {
"total": 36,
"successful": 36,
"failed": 0
},
"hits": {
"total": 718,
"max\_score": 0.0,
"hits": []
},
"aggregations": {
"2": {
"doc\_count\_error\_upper\_bound": 0,
"sum\_other\_doc\_count": 0,
"buckets": [{
"key": "select (_) from t\_intg\_dm\_00v3 where XXX = XXX and ((((((fm\_office in (XXX) and task\_type in (XXX)) and task\_status != XXX) and task\_status != XXX) and task\_status != XXX) and auto\_schedulable != XXX) and change\_time \> XXX) and active = XXX limit XXX, XXX;",
"doc\_count": 12
},
{
"key": "select (_) from t\_intg\_dm\_00v3 where XXX = XXX and task\_id = 'XX-XXXXXXX-XXXXX' and active = XXX order by operate\_time asc limit XXX, XXX;",
"doc\_count": 2
},
{
"key": "select (_) from t\_intg\_dm\_00v3 where XXX = XXX and (task\_id = 'XX-XXXXXXX-XXXXX' and (operate\_type = XXX or operate\_type = XXX)) and active = XXX order by task\_log\_id asc limit XXX, XXX;",
"doc\_count": 1
},
{
"key": "select (_) from t\_intg\_dm\_00v3 where XXX = XXX and (task\_id = 'XX-XXXXXXX-XXXXX' and (operate\_type = XXX or operate\_type = XXX)) and active = XXX order by task\_log\_id asc limit XXX, XXX;",
"doc\_count": 1
},
{
"key": "select count(1) count from t\_intg\_dm\_00ui where XXX = XXX and create\_time \<= '2016-08-15 XXX:59:59' and active = XXX;",
"doc\_count": 1
}]
}
}
}
---
### Author:  [@cbuescher](https://discuss.elastic.co/u/cbuescher)
#### Post date: [August 15, 2016, 1:12pm UTC](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024/2 "2016-08-15T13:12:52Z")
Whats the mapping for the field you are aggregating on?
---
### Author:  [@pill663](https://discuss.elastic.co/u/pill663)
#### Post date: [August 16, 2016, 2:43am UTC](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024/3 "2016-08-16T02:43:16Z")
the name of mapping is 'slowquery'.
it has many keys, such as 'Pattern','Database','Schema'.
I'm doing term aggregation on the key 'Pattern'. It's string.
---
### Author:  [@cbuescher](https://discuss.elastic.co/u/cbuescher)
#### Post date: [August 16, 2016, 8:49am UTC](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024/4 "2016-08-16T08:49:15Z")
From your example its hard to tell, can you add how you are doing the aggregation?
---
### Author:  [@pill663](https://discuss.elastic.co/u/pill663)
#### Post date: [August 16, 2016, 10:58am UTC](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024/5 "2016-08-16T10:58:36Z")
My query body is like this,the field is 'Pattern',mapping is 'slowquery':
{
"query": {
"filtered": {
"query": {
"query\_string": {
"query": "type:slowquery",
"analyze\_wildcard": true
}
},
"filter": {
"bool": {
"must": [
{
"range": {
"@timestamp": {
"gte": 1471299986398,
"lte": 1471314386399,
"format": "epoch\_millis"
}
}
}
],
"must\_not": []
}
}
}
},
"size": 0,
"aggs": {
"2": {
"terms": {
"field": "Pattern.raw",
"size": 500,
"order": {
"\_count": "desc"
}
}
}
}
}
---
### Author:  [@cbuescher](https://discuss.elastic.co/u/cbuescher)
#### Post date: [August 16, 2016, 12:40pm UTC](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024/6 "2016-08-16T12:40:00Z")
If you run the query without the aggregation (and size set to some higher value), do all the results have the field set that you are aggregating on?
---
### Author:  [@pill663](https://discuss.elastic.co/u/pill663)
#### Post date: [August 17, 2016, 1:46am UTC](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024/7 "2016-08-17T01:46:09Z")
If I query without aggregation, then the results number is correct.
such as, curl -XGET '[http://esurl/\_all/\_search?size=200](http://esurl/_all/_search?size=200)' then the return size is correct.
when I add the aggregation query body, the result is much less.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [August 17, 2016, 5:38am UTC](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024/8 "2016-08-17T05:38:46Z")
Please provide the mapping for the field. You can retrieve this through the [get mapping API](https://www.elastic.co/guide/en/elasticsearch/reference/2.3/indices-get-mapping.html#indices-get-mapping).
---
### Author:  [@pill663](https://discuss.elastic.co/u/pill663)
#### Post date: [August 17, 2016, 7:04am UTC](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024/9 "2016-08-17T07:04:54Z")
{"logstash-2016.08.16":{"mappings":{"slowquery":{"\_all":{"enabled":true,"omit\_norms":true},"dynamic\_templates":[{"message\_field":{"mapping":{"fielddata":{"format":"disabled"},"index":"analyzed","omit\_norms":true,"type":"string"},"match":"message","match\_mapping\_type":"string"}},{"string\_fields":{"mapping":{"fielddata":{"format":"disabled"},"index":"analyzed","omit\_norms":true,"type":"string","fields":{"raw":{"ignore\_above":256,"index":"not\_analyzed","type":"string","doc\_values":true}}},"match":"_","match\_mapping\_type":"string"}},{"float\_fields":{"mapping":{"type":"float","doc\_values":true},"match":"_","match\_mapping\_type":"float"}},{"double\_fields":{"mapping":{"type":"double","doc\_values":true},"match":"_","match\_mapping\_type":"double"}},{"byte\_fields":{"mapping":{"type":"byte","doc\_values":true},"match":"_","match\_mapping\_type":"byte"}},{"short\_fields":{"mapping":{"type":"short","doc\_values":true},"match":"_","match\_mapping\_type":"short"}},{"integer\_fields":{"mapping":{"type":"integer","doc\_values":true},"match":"_","match\_mapping\_type":"integer"}},{"long\_fields":{"mapping":{"type":"long","doc\_values":true},"match":"_","match\_mapping\_type":"long"}},{"date\_fields":{"mapping":{"type":"date","doc\_values":true},"match":"_","match\_mapping\_type":"date"}},{"geo\_point\_fields":{"mapping":{"type":"geo\_point","doc\_values":true},"match":"\*","match\_mapping\_type":"geo\_point"}}],"properties":{"@timestamp":{"type":"date","format":"strict\_date\_optional\_time||epoch\_millis"},"@version":{"type":"string","index":"not\_analyzed"},"Database":{"type":"string","norms":{"enabled":false},"fielddata":{"format":"disabled"},"fields":{"raw":{"type":"string","index":"not\_analyzed","ignore\_above":256}}},"Lock Time":{"type":"double"},"Pattern":{"type":"string","norms":{"enabled":false},"fielddata":{"format":"disabled"},"fields":{"raw":{"type":"string","index":"not\_analyzed","ignore\_above":256}}},"Query Time":{"type":"double"},"Rows Examined":{"type":"long"},"Rows Sent":{"type":"long"},"SQL":{"type":"string","norms":{"enabled":false},"fielddata":{"format":"disabled"},"fields":{"raw":{"type":"string","index":"not\_analyzed","ignore\_above":256}}},"Schema":{"type":"string","norms":{"enabled":false},"fielddata":{"format":"disabled"},"fields":{"raw":{"type":"string","index":"not\_analyzed","ignore\_above":256}}},"Timestamp":{"type":"date","format":"strict\_date\_optional\_time||epoch\_millis"},"beat":{"properties":{"hostname":{"type":"string","norms":{"enabled":false},"fielddata":{"format":"disabled"},"fields":{"raw":{"type":"string","index":"not\_analyzed","ignore\_above":256}}},"name":{"type":"string","norms":{"enabled":false},"fielddata":{"format":"disabled"},"fields":{"raw":{"type":"string","index":"not\_analyzed","ignore\_above":256}}}}},"count":{"type":"long"},"geoip":{"dynamic":"true","properties":{"ip":{"type":"ip"},"latitude":{"type":"float"},"location":{"type":"geo\_point"},"longitude":{"type":"float"}}},"host":{"type":"string","norms":{"enabled":false},"fielddata":{"format":"disabled"},"fields":{"raw":{"type":"string","index":"not\_analyzed","ignore\_above":256}}},"input\_type":{"type":"string","norms":{"enabled":false},"fielddata":{"format":"disabled"},"fields":{"raw":{"type":"string","index":"not\_analyzed","ignore\_above":256}}},"log\_format":{"type":"string","norms":{"enabled":false},"fielddata":{"format":"disabled"},"fields":{"raw":{"type":"string","index":"not\_analyzed","ignore\_above":256}}},"message":{"type":"string","norms":{"enabled":false},"fielddata":{"format":"disabled"}},"offset":{"type":"long"},"source":{"type":"string","norms":{"enabled":false},"fielddata":{"format":"disabled"},"fields":{"raw":{"type":"string","index":"not\_analyzed","ignore\_above":256}}},"tags":{"type":"string","norms":{"enabled":false},"fielddata":{"format":"disabled"},"fields":{"raw":{"type":"string","index":"not\_analyzed","ignore\_above":256}}},"type":{"type":"string","norms":{"enabled":false},"fielddata":{"format":"disabled"},"fields":{"raw":{"type":"string","index":"not\_analyzed","ignore\_above":256}}}}}}}}
---
### Author:  [@pill663](https://discuss.elastic.co/u/pill663)
#### Post date: [August 17, 2016, 7:05am UTC](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024/10 "2016-08-17T07:05:53Z")
It's a little bit long , so I didn't transfer it to json style.
---
### Author:  [@pill663](https://discuss.elastic.co/u/pill663)
#### Post date: [August 17, 2016, 7:08am UTC](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024/11 "2016-08-17T07:08:01Z")
It seems the field has one attribute, "ignore\_above": 256.
Is it cause the problem. if string is longer than 256,it will be ignored when doing aggregation?
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [August 17, 2016, 7:13am UTC](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024/12 "2016-08-17T07:13:50Z")
I believe it will only index the first 256 bytes of the not\_analyzed string, which could explain why expressions that differ only after the 256th byte gets grouped together.
---
### Author:  [@pill663](https://discuss.elastic.co/u/pill663)
#### Post date: [August 17, 2016, 7:15am UTC](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024/13 "2016-08-17T07:15:05Z")
How to change it , such as to 1024 bytes.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [August 17, 2016, 7:25am UTC](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024/14 "2016-08-17T07:25:51Z")
You will need to change this in your [index template](https://www.elastic.co/guide/en/elasticsearch/reference/2.3/indices-templates.html).
---
### Author:  [@pill663](https://discuss.elastic.co/u/pill663)
#### Post date: [August 17, 2016, 10:08am UTC](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024/15 "2016-08-17T10:08:18Z")
I'm using logstash to send data to es.
It seems that index template has no impact on existing indices.
If I want to change the existing indices, what can I do?
---
### Author:  [@pill663](https://discuss.elastic.co/u/pill663)
#### Post date: [August 17, 2016, 10:10am UTC](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024/16 "2016-08-17T10:10:02Z")
And when I tried to update mappings,
curl -XPUT '[http://myesurl/logstash-2016.08.16/\_mapping/slowquery](http://myesurl/logstash-2016.08.16/_mapping/slowquery)' -d '
"properties": {
"Pattern": {
"type": "string",
"norms": {
"enabled": false
},
"fielddata": {
"format": "disabled"
},
"fields": {
"raw": {
"type": "string",
"index": "not\_analyzed",
"ignore\_above": 2560
}
}
},
"SQL": {
"type": "string",
"norms": {
"enabled": false
},
"fielddata": {
"format": "disabled"
},
"fields": {
"raw": {
"type": "string",
"index": "not\_analyzed",
"ignore\_above": 2560
}
}
}
}
'
It returned error:
{"error":{"root\_cause":[{"type":"not\_x\_content\_exception","reason":"not\_x\_content\_exception: Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"}],"type":"not\_x\_content\_exception","reason":"not\_x\_content\_exception: Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"},"status":500}
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [August 17, 2016, 10:18am UTC](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024/17 "2016-08-17T10:18:41Z")
You can not change existing mappings, so you will need to reindex that data.
---
### Author:  [@pill663](https://discuss.elastic.co/u/pill663)
#### Post date: [August 17, 2016, 11:18am UTC](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024/18 "2016-08-17T11:18:14Z")
the data is send to es in json format by logstash.
If the mapping was not exist,es will create it automatically according to the structure of json string.
So I think , is it possible to configure the value of ignore\_above in logstash data,so that the mapping will be correctlly created at the very start.
---
### Author:  [@pill663](https://discuss.elastic.co/u/pill663)
#### Post date: [August 17, 2016, 11:20am UTC](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024/19 "2016-08-17T11:20:11Z")
And the index is created by day, does index template impact on the slowquery mapping of all indices?
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [August 17, 2016, 11:42am UTC](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024/20 "2016-08-17T11:42:41Z")
Any changes to the index template will only apply for new indices that are created. It will not affect existing ones, so the data in those may need to be reindexed.
[Next page](https://discuss.elastic.co/t/is-there-any-length-limitation-of-the-key-in-term-aggregation-i-mean-results-display/58024.md?page=2)