# Composite aggregation max bucket size

**URL:** <https://discuss.elastic.co/t/composite-aggregation-max-bucket-size/155663>\
**Category:** Elasticsearch\
**Created:** [November 7, 2018, 6:56am UTC](https://discuss.elastic.co/t/composite-aggregation-max-bucket-size/155663 "2018-11-07T06:56:33Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![blueren](https://avatars.discourse-cdn.com/v4/letter/b/ecae2f/32.png) [@blueren](https://discuss.elastic.co/u/blueren)\
**Post date:** [November 7, 2018, 6:56am UTC](https://discuss.elastic.co/t/composite-aggregation-max-bucket-size/155663/1 "2018-11-07T06:56:33Z")

</div>

I have a query that I need to run every 30s to check if there are any docs matching the params.

```
{
  "version": true,
  "size": 0,
  "sort": [
    {
      "@timestamp": {
        "order": "desc",
        "unmapped_type": "boolean"
      }
    }
  ],
  "_source": {
    "excludes": []
  },
  "aggs": {
    "my_buckets": {
      "composite": {
        "size": 2147483647, <------ causes cirtuit braker exception
        "sources": [
          {
            "blk_srvPort": {
              "terms": {
                "field": "flow.service_port"
              }
            }
          },
          {
            "src_addr": {
              "terms": {
                "field": "flow.src_addr"
              }
            }
          },
          {
            "dst_addr": {
              "terms": {
                "field": "flow.dst_addr"
              }
            }
          },
          {
            "es_id": {
              "terms": {
                "field": "_id"
              }
            }
          },
          {
            "es_timestamp": {
              "terms": {
                "field": "@timestamp"
              }
            }
          }
        ]
      }
    }
  },
  "stored_fields": [
    "*"
  ],
  "script_fields": {},
  "docvalue_fields": [
    "@timestamp",
    "netflow.first_switched",
    "netflow.last_switched"
  ],
  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "query": """flow.dst_addr: "10.5.6.25" """,
            "analyze_wildcard": true,
            "default_field": "*"
          }
        },
        {
          "range": {
            "@timestamp": {
              "gte": "now-10s",
              "lte": "now"
            }
          }
        }
      ],
      "filter": [],
      "should": [],
      "must_not": []
    }
  }
}

```

I'm using composite aggregation since I need to know multiple fields inside of each bucket - this works well. However, how do I get all the buckets and make sure none of them are dropped due to predefined limits? I know there is a "size" parameter that can be used inside composite, but what is the max possible value that I can set to the size property? Is there a way I force it to return ALL buckets?

---

<div class="post-metadata">

**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\
**Post date:** [November 12, 2018, 3:42pm UTC](https://discuss.elastic.co/t/composite-aggregation-max-bucket-size/155663/2 "2018-11-12T15:42:58Z")

</div>

The `size` parameter of the composite agg is just how many buckets you want _per page_. The composite agg "paginates" over all the buckets, so it is an exhaustive aggregation that will return all the results once you've fully paginated through it.

See the docs here about specifying an `after` parameter to keep paging: [https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-composite-aggregation.html#\_after](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-composite-aggregation.html#_after)

So basically, you set the size to something that is a reasonable balance between speed (higher size) and memory requirements (lower size). Then you page through the results fully with multiple requests.

Hope that helps!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 10, 2018, 3:42pm UTC](https://discuss.elastic.co/t/composite-aggregation-max-bucket-size/155663/3 "2018-12-10T15:42:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.