# Bulk\_max\_body\_size support?

**URL:** <https://discuss.elastic.co/t/bulk-max-body-size-support/76611>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [February 27, 2017, 11:27am UTC](https://discuss.elastic.co/t/bulk-max-body-size-support/76611 "2017-02-27T11:27:54Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![PicoCreator](https://avatars.discourse-cdn.com/v4/letter/p/a6a055/32.png) [@PicoCreator](https://discuss.elastic.co/u/PicoCreator)\
**Post date:** [February 27, 2017, 11:27am UTC](https://discuss.elastic.co/t/bulk-max-body-size-support/76611/1 "2017-02-27T11:27:54Z")

</div>

An alternative limit to bulk\_max\_size.  
That functions based on the payload size instead.

```auto
output:

  ### Elasticsearch as output
  elasticsearch:
    # Array of hosts to connect to.
    hosts: ["${ES_HOST}:${ES_PORT}"]

    # The maximum size to send in a single Elasticsearch bulk API index request.
    bulk_max_body_size: 10M

    # The maximum number of events to bulk in a single Elasticsearch bulk API index request.
    bulk_max_size: 50

```

This limitation is required due to managed Elasticsearch deployments (such as AWS)  
having upload size limits of 10 MB for entry level.

See: [http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/aes-limits.html](http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/aes-limits.html)

Because a single multiline message caps at 10MB by default, with 50 for batch processing.  
The current "limit" is about 500MB with some overheads.

Currently when this happen a 413 error is perpetually repeated. Specifically the following.

```auto
client.go:244: ERR Failed to perform any bulk index operations: 413 Request Entity Too Large

```

As there is no way to increase the limit on AWS side, nor on the filebeat side,  
other then to greatly decrease the max log size, and bulk\_max\_size.

This greatly limit the configuration options in such situations.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [February 27, 2017, 11:38pm UTC](https://discuss.elastic.co/t/bulk-max-body-size-support/76611/2 "2017-02-27T23:38:39Z")

</div>

This is currently not supported, but agreed, this features totally makes sense to have (it's a MUST). Feel free to open an [enhancement request](https://github.com/elastic/beats/issues).

---

<div class="post-metadata">

**Author:** ![PicoCreator](https://avatars.discourse-cdn.com/v4/letter/p/a6a055/32.png) [@PicoCreator](https://discuss.elastic.co/u/PicoCreator)\
**Post date:** [February 28, 2017, 2:36am UTC](https://discuss.elastic.co/t/bulk-max-body-size-support/76611/3 "2017-02-28T02:36:23Z")

</div>

Sure, filed as : [https://github.com/elastic/beats/issues/3688](https://github.com/elastic/beats/issues/3688)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 28, 2017, 2:37am UTC](https://discuss.elastic.co/t/bulk-max-body-size-support/76611/4 "2017-03-28T02:37:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.