Hi,
I have made a control visualization (option list) with 4 options-fields (let's say A, B, C & D).
The "size" field of each option is 10000 and search.max_buckets setting is 10000 too.
In ElasticSearch, for field A there are 2 distinguished values , A1 & A2.
The documents with A1 value in A field are 14800000 and they have 1028 distinguished values in field B.
The documents with A2 value in A field are 3700000 and they have 3500 distinguished values in field B.
In control visualization, when I select for field A the A1 value I get the error "5 of 12 shards failed" and when I inspect the query and the response to elasticsearch I see that
"type": "too_many_buckets_exception",
"reason": "Trying to create too many buckets. Must be less than or equal to: [10000] but was [10001]. This limit can be set by changing the [search.max_buckets] cluster level setting.",
But when I select A2 everything is ok.
Why the error is not shown with value A2?
Is this because, with a simple calculation, the unique buckets for A1 value are14800000 : 1028 = 14396,.... and greater than max_bucket or "size"
and for A2 are 3700000 : 3500 = 1057,.... ??
Also, I saw that *max_buckets = size 1.5 + 10 in https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#_shard_size_3
That means that "size" should be set max to 6660 if max_bucket = 10000 ?
If I set size to 6000 there is no problem but if I set to 7000 I see againe the too_many_buckets_exception for A1.
Which value , max_bucket or "size" should I change in order not to get exceptions? Which calculations should I do?
Thank you in advance!
BR
Paraskevi