# How to tuning terms aggregation performance
**URL:** https://discuss.elastic.co/t/how-to-tuning-terms-aggregation-performance/184907
**Category:** Elasticsearch
**Created:** [June 10, 2019, 2:36am UTC](https://discuss.elastic.co/t/how-to-tuning-terms-aggregation-performance/184907 "2019-06-10T02:36:00Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@kkd927](https://discuss.elastic.co/u/kkd927)
#### Post date: [June 10, 2019, 2:36am UTC](https://discuss.elastic.co/t/how-to-tuning-terms-aggregation-performance/184907/1 "2019-06-10T02:36:00Z")
Hi there
We have a problem with our ES terms aggregation query, it took 10-12s to execute.
here is our cluster information
1. we have 3 client nodes, 3 master nodes, 5 data nodes, 1 ingest node
2. each node, its 20 cores(40vCore) and 64GB memeory, we assigned 31GB to the heap
3. index have 5 shards and 1 replica
4. index size is around 340GB (primary - 165GB) and document size is around 934.4m
5. ES version is 6.5.4
index have document's tag information.
```
{
"mapping": {
"_doc": {
"_field_names": {
"enabled": false
},
"properties": {
"blogId": {
"type": "keyword"
},
"tag": {
"type": "keyword",
"boost": 30,
"eager_global_ordinals": true,
"copy_to": [
"tagNgram"
]
},
"tagNgram": {
"type": "text",
"analyzer": "ngram_analyzer",
"search_analyzer": "standard"
}
}
}
}
}
```
data seems like:
```
{
"blogId": "00001",
"tag": "APPLE"
},
{
"blogId": "00001",
"tag": "BANANA"
},
{
"blogId": "00001",
"tag": "ORANGE"
},
{
"blogId": "00002",
"tag": "APPLE"
},
{
"blogId": "00003",
"tag": "PEACH"
},
{
"blogId": "00003",
"tag": "BANANA"
}
```
here is my query
```
GET /tag_search_index/_doc/_search
{
"size": 0,
"query": {
"bool": {
"filter": {
"match": { "tagNgram": "A" }
},
"must_not": {
"term": { "tag": "A" }
}
}
},
"aggs": {
"most_popular": {
"terms": {
"field": "tag",
"size": 10
}
},
"count":{
"cardinality": {
"field": "tag"
}
}
}
}
```
response is
```
{
"took" : 12380,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 64946917,
"max_score" : 0.0,
"hits" : []
},
"aggregations" : {
"count" : {
"value" : 7202919
},
"most_popular" : {
"doc_count_error_upper_bound" : 46346,
"sum_other_doc_count" : 61546148,
"buckets" : [
// ...
]
}
}
}
```
Searching with more than two characters speeds up your search. But if searching with one letter, it takes about 10 seconds.
Is there any way to make it faster?
thanks in advance.
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [June 10, 2019, 6:43am UTC](https://discuss.elastic.co/t/how-to-tuning-terms-aggregation-performance/184907/2 "2019-06-10T06:43:19Z")
You could try set the “collect mode” on the terms aggregation to breadth-first.
I’m not sure if that’s automatically picked for this particular request but enabling it means we compute the top 10 tags first before computing their child cardinality aggs (as opposed to calculating all cardinalities then pruning to top 10)
---
### Author:  [@kkd927](https://discuss.elastic.co/u/kkd927)
#### Post date: [June 10, 2019, 7:04am UTC](https://discuss.elastic.co/t/how-to-tuning-terms-aggregation-performance/184907/3 "2019-06-10T07:04:11Z")
In the query I'm calling, cardinality aggs is not child of terms aggs.
```
"aggs": {
"most_popular": {
"terms": {
"field": "tag",
"size": 10
}
},
"count":{
"cardinality": {
"field": "tag"
}
}
}
```
I want to aggregate the number of all tags that contain A characters, and separately check the top 10 tags.
I understand that collect mode is only available in the aggs of parent-child relationships, am I right?
(Please understand that I am not good at English.)
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [June 10, 2019, 8:13am UTC](https://discuss.elastic.co/t/how-to-tuning-terms-aggregation-performance/184907/4 "2019-06-10T08:13:58Z")
> [@kkd927](#):
>
> In the query I'm calling, cardinality aggs is not child of terms aggs.
Ah. My bad.
> [@kkd927](#):
>
> I want to aggregate the number of all tags that contain A characters, and separately check the top 10 tags.
Do blogs have multiple tags? If so then your counts and top tens could consist of things that co-occur with A\* tags rather than just being A\* tags
---
### Author:  [@kkd927](https://discuss.elastic.co/u/kkd927)
#### Post date: [June 10, 2019, 9:39am UTC](https://discuss.elastic.co/t/how-to-tuning-terms-aggregation-performance/184907/5 "2019-06-10T09:39:30Z")
> [@Mark\_Harwood](#):
>
> Do blogs have multiple tags?
yes, each blog have mulitple tags.
00001 blog : APPLE, BANANA, ORANGE
00002 blog : APPLE
00003 blog : PEACH, BANANA
...
> [@Mark\_Harwood](#):
>
> co-occur with A\* tags
what means "co-occure with A\* tags"?
I want to get result following:
1. counts of all tags that contain A characters.
2. Top N tags that contain A characters.
So this is the query that I want.
```
GET /tag_search_index/_doc/_search
{
"size": 0,
"query": {
"bool": {
"filter": {
"match": { "tagNgram": "A" }
},
"must_not": {
"term": { "tag": "A" }
}
}
},
"aggs": {
"most_popular": {
"terms": {
"field": "tag",
"size": 10
}
},
"count":{
"cardinality": {
"field": "tag"
}
}
}
}
```
result is
```
"aggregations" : {
"count" : {
// counts of all tags that contain A characters.
"value" : 7202919
},
"most_popular" : {
"doc_count_error_upper_bound" : 46346,
"sum_other_doc_count" : 61546148,
"buckets" : [
// Top N tags that contain A characters.
{
"key": "APPLE",
"doc_count": "15667"
},
{
"key": "BANANA",
"doc_count": "11491"
},
// ...
]
}
```
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [June 10, 2019, 4:14pm UTC](https://discuss.elastic.co/t/how-to-tuning-terms-aggregation-performance/184907/6 "2019-06-10T16:14:19Z")
Queries serve only to filter documents - not the values that appear in those documents.
The aggregations work on _all_ the values in the filtered documents. You can use an ‘include’ clause with a regular expression inside the ‘terms’ aggregation to consider only tags that start with A.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 8, 2019, 4:21pm UTC](https://discuss.elastic.co/t/how-to-tuning-terms-aggregation-performance/184907/7 "2019-07-08T16:21:35Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.