# Overcoming search.max\_buckets Limitation in AWS Elasticsearch for Shard-Level Aggregations

**URL:** <https://discuss.elastic.co/t/overcoming-search-max-buckets-limitation-in-aws-elasticsearch-for-shard-level-aggregations/358079>\
**Category:** Elasticsearch\
**Created:** [April 24, 2024, 5:05am UTC](https://discuss.elastic.co/t/overcoming-search-max-buckets-limitation-in-aws-elasticsearch-for-shard-level-aggregations/358079 "2024-04-24T05:05:50Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Yuki\_Hara](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuki_hara/32/133796_2.png) [@Yuki\_Hara](https://discuss.elastic.co/u/Yuki_Hara)\
**Post date:** [April 24, 2024, 5:05am UTC](https://discuss.elastic.co/t/overcoming-search-max-buckets-limitation-in-aws-elasticsearch-for-shard-level-aggregations/358079/1 "2024-04-24T05:05:50Z")

</div>

**Objective:**

Perform aggregation (terms aggregation) on all documents within each shard.

**Actions Taken & Issues:**

- Set `size: 100` (assuming `size` should remain unchanged)
- Set `shard_size` to 2147483519
- Set `shard_size` to 10000

**Result:**

The following error occurs:

`Trying to create too many buckets. Must be less than or equal to: [65535] but was [65544]. This limit can be set by changing the [search.max_buckets] cluster level setting.`

However, the Elasticsearch instance is hosted on AWS (Amazon OpenSearch Service), and modifying `search.max_buckets` is not possible. [https://docs.aws.amazon.com/AmazonS3/latest/userguide/BucketRestrictions.html](https://docs.aws.amazon.com/AmazonS3/latest/userguide/BucketRestrictions.html)

**Elasticsearch Information:**

- Version: 7.10
- Shards

```auto
JSON[
  {
    "index": "index_name",
    "shard": "0",
    "prirep": "p",
    "state": "STARTED",
    "docs": "50031782",
    "store": "39.3gb",
    ...
  },
  {
    "index": "index_name",
    "shard": "1",
    "prirep": "p",
    "state": "STARTED",
    "docs": "49976007",
    "store": "39.2gb",
    ...
  },
  {
    "index": "index_name",
    "shard": "2",
    "prirep": "p",
    "state": "STARTED",
    "docs": "49976709",
    "store": "39.3gb",
    ...
  },
  {
    "index": "index_name",
    "shard": "3",
    "prirep": "p",
    "state": "STARTED",
    "docs": "49998407",
    "store": "39.3gb",
    ...
  },
  {
    "index": "index_name",
    "shard": "4",
    "prirep": "p",
    "state": "STARTED",
    "docs": "49971995",
    "store": "39.2gb",
    ...
  }
]

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 24, 2024, 5:05am UTC](https://discuss.elastic.co/t/overcoming-search-max-buckets-limitation-in-aws-elasticsearch-for-shard-level-aggregations/358079/2 "2024-04-24T05:05:50Z")

</div>

OpenSearch/OpenDistro are AWS run products and differ from the original Elasticsearch and Kibana products that Elastic builds and maintains. You may need to contact them directly for further assistance. See [What is OpenSearch and the OpenSearch Dashboard? | Elastic](https://www.elastic.co/elasticsearch/opensearch) for more details.

(This is an automated response from your friendly Elastic bot. Please report this post if you have any suggestions or concerns :elasticheart: )

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [April 24, 2024, 5:21am UTC](https://discuss.elastic.co/t/overcoming-search-max-buckets-limitation-in-aws-elasticsearch-for-shard-level-aggregations/358079/3 "2024-04-24T05:21:20Z")

</div>

This is something you will need to discuss with AWS support as it is a restriction in their service.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 24, 2024, 5:23am UTC](https://discuss.elastic.co/t/overcoming-search-max-buckets-limitation-in-aws-elasticsearch-for-shard-level-aggregations/358079/4 "2024-04-24T05:23:50Z")

</div>

BTW did you look at [Cloud by Elastic](https://www.elastic.co/cloud), also available if needed from [AWS Marketplace](https://aws.amazon.com/marketplace/pp/Elasticsearch-Inc-Elasticsearch-Service-on-Elastic/B01N6YCISK), [Azure Marketplace](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/elastic.ec-azure?tab=Overview) and [Google Cloud Marketplace](https://console.cloud.google.com/marketplace/details/endpoints/elasticsearch-service.gcpmarketplace.elastic.co)?

Cloud by elastic is one way to have access to **all features** , all managed by us. Think about what is there yet like Security, Monitoring, Reporting, SQL, Canvas, Maps UI, Alerting and built-in solutions named [Observability](https://www.elastic.co/observability), [Security](https://www.elastic.co/security), [Enterprise Search](https://www.elastic.co/enterprise-search) and what is coming next 🙂 ...