I'm getting this error in our some of our Grafana panels now after updating to 7.4 from 6.8:
type:"too_many_buckets_exception"
reason:"Trying to create too many buckets. Must be less than or equal to: [10000] but was [10001]. This limit can be set by changing the [search.max_buckets] cluster level setting."
Elastic ref says:
The maximum number of buckets allowed in a single response is limited by a dynamic cluster setting namedsearch.max_buckets. It defaults to 10,000, requests that try to return more than the limit will fail with an exception
Seeing this issue also after updating to 7.x, mentioning: 'So be careful about making the limit too high'
What might be considered too high, when nodes trips over with OOM or...?
You should decrease time period or increase group interval for charts in order to decrease buckets count. Also Grafana should use composite aggregator or scroll API
time period is hard to decrease as grafana users can select a 'large' time periods to view many data points.
Something most have changed in 7.x vs. previous version as these specific grafana panels have always worked with default search.max_buckets in version 2.x, 5.x and 6.x.
Now I've increased search.max_buckets to > 35000 then panels seems to work again also showing larger time periods.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.