# Is there a way to batch your query when facing with too\_many\_buckets\_exception
**URL:** https://discuss.elastic.co/t/is-there-a-way-to-batch-your-query-when-facing-with-too-many-buckets-exception/214293
**Category:** Elasticsearch
**Created:** [January 8, 2020, 7:08pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-batch-your-query-when-facing-with-too-many-buckets-exception/214293 "2020-01-08T19:08:27Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@Tomaz\_Bratanic1](https://discuss.elastic.co/u/Tomaz_Bratanic1)
#### Post date: [January 8, 2020, 7:08pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-batch-your-query-when-facing-with-too-many-buckets-exception/214293/1 "2020-01-08T19:08:27Z")
I use the following query:
```
{
"aggs": {
"2": {
"terms": {
"field": "deviceID.keyword",
"order": {
"_key": "desc"
},
"size": 500000
},
"aggs": {
"1": {
"top_hits": {
"_source": "deviceIP",
"size": 1
}
}
}
}
},
"size": 0,
"_source": {
"excludes": []
},
"stored_fields": [
"*"
],
"script_fields": {},
"docvalue_fields": [
{
"field": "@timestamp",
"format": "date_time"
}
],
"query": {
"bool": {
"must": [
{
"range": {
"@timestamp": {
"gte": "now-30d"
}
}
}
],
"filter": [
{
"match_all": {}
}
],
"should": [],
"must_not": []
}
}
]
}
}
}
```
I am getting the following exception, which is quite self-explanatory.
```
"reason" : {
"type" : "too_many_buckets_exception",
"reason" : "Trying to create too many buckets. Must be less than or equal to: [10000] but was [10001]. This limit can be set by changing the [search.max_buckets] cluster level setting.",
"max_buckets" : 10000
}
```
Can I basically split the query into batches, so that I don't have to increase the max\_buckets setting? Or is the only way to get the IP for like 50k devices to increase the max\_buckets setting?
---
### Author:  [@wylie](https://discuss.elastic.co/u/wylie)
#### Post date: [January 8, 2020, 7:20pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-batch-your-query-when-facing-with-too-many-buckets-exception/214293/2 "2020-01-08T19:20:09Z")
It's a bit of a different query, but this is supported by the Composite aggregation. [https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-composite-aggregation.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-composite-aggregation.html)
The composite aggregation is limited in functionality because it doesn't accept sorting by metrics, but you are already sorting by `_key` which is the default behavior of the composite aggregation.
Within each Term in your composite aggregation you can nest your `top_hits` aggregation.
---
### Author:  [@Tomaz\_Bratanic1](https://discuss.elastic.co/u/Tomaz_Bratanic1)
#### Post date: [January 8, 2020, 7:29pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-batch-your-query-when-facing-with-too-many-buckets-exception/214293/3 "2020-01-08T19:29:33Z")
Thanks for the input!
As I am a bit of a ES noob, this is all very advanced to me and I don't know how to begin. Could you give me a hint of what would my final query look like?
---
### Author:  [@Mikhail\_Khludnev](https://discuss.elastic.co/u/Mikhail_Khludnev)
#### Post date: [January 8, 2020, 7:31pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-batch-your-query-when-facing-with-too-many-buckets-exception/214293/4 "2020-01-08T19:31:40Z")
Hello,
I think you can page aggregations via
[https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-composite-aggregation.html#\_after](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-composite-aggregation.html#_after)
---
### Author:  [@wylie](https://discuss.elastic.co/u/wylie)
#### Post date: [January 8, 2020, 7:43pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-batch-your-query-when-facing-with-too-many-buckets-exception/214293/5 "2020-01-08T19:43:12Z")
Per the documentation, this is an example:
```auto
GET /_search
{
"aggs" : {
"my_buckets": {
"composite" : {
"sources" : [
{ "date": { "date_histogram": { "field": "timestamp", "calendar_interval": "1d", "order": "desc" } } },
{ "product": { "terms": {"field": "product" } } }
]
},
"aggregations": {
"the_avg": {
"avg": { "field": "price" }
}
}
}
}
}
```
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 5, 2020, 7:43pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-batch-your-query-when-facing-with-too-many-buckets-exception/214293/6 "2020-02-05T19:43:25Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.