Allowing size to be max [possibly inifinity]

Elastic Stack Elasticsearch
1

Hi, I've the below an aggregate query response:
As you can see "sum_other_doc_count": 399 and only 10 buckets are returned. I would like sum_other_doc_count to be always 0 and return all buckets, irrespective of time it takes to perform the query. Is there any way?

{
    "took": 18,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 82,
            "relation": "eq"
        },
        "max_score": null,
        "hits": []
    },
    "aggregations": {
        "aq_agg_test_case": {
            "doc_count": 474,
            "aq_agg_name": {
                "doc_count_error_upper_bound": 0,
                "sum_other_doc_count": 399,
                "buckets": [
                    {
                        "key": "Resource Management - TC-1",
                        "doc_count": 8
                    },
                    {
                        "key": "Resource Management - TC-2",
                        "doc_count": 8
                    },
                    {
                        "key": "Resource Management - TC-3",
                        "doc_count": 8
                    },
                    {
                        "key": "Resource Management - TC-4",
                        "doc_count": 8
                    },
                    {
                        "key": "Resource Management - TC-5",
                        "doc_count": 8
                    },
                    {
                        "key": "Collaboration Features - TC-1",
                        "doc_count": 7
                    },
                    {
                        "key": "Collaboration Features - TC-2",
                        "doc_count": 7
                    },
                    {
                        "key": "Cross-Device Syncing-TC-1",
                        "doc_count": 7
                    },
                    {
                        "key": "Cross-Device Syncing-TC-2",
                        "doc_count": 7
                    },
                    {
                        "key": "Cross-Device Syncing-TC-3",
                        "doc_count": 7
                    }
                ]
            }
        }
    }
}
2

Hi,

you can increase the size parameter in your aggregation query. However, please note that setting a very high value for the size parameter can significantly increase the time it takes to perform the query and the amount of resources used.

Here is an example of how to increase the size parameter in your aggregation:

{
  "aggs": {
    "aq_agg_test_case": {
      "terms": {
        "field": "your_field",
        "size": 10000  // Increase this value to get more buckets
      }
    }
  }
}

Regards

3

Ok, so one question. Is the max limit of size value to 10,000? Or I can put any value in it?

4

Hi,

The maximum limit for the size parameter in Elasticsearch terms aggregation is indeed 10,000. This limit is set to prevent memory issues.

5

So let's say at some point of time the limit prevents all aggregation results to be retrieved. So is there any way to iterate over the rest of aggregation buckets?

Meaning i can somehow iterate over sum_other_doc_count maybe making multiple requests with some pagination?