# ContentTooLongException when using Reindex API

**URL:** <https://discuss.elastic.co/t/contenttoolongexception-when-using-reindex-api/64471>\
**Category:** Elasticsearch\
**Created:** [October 31, 2016, 5:28pm UTC](https://discuss.elastic.co/t/contenttoolongexception-when-using-reindex-api/64471 "2016-10-31T17:28:55Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![hariso](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hariso/32/50668_2.png) [@hariso](https://discuss.elastic.co/u/hariso)\
**Post date:** [October 31, 2016, 5:28pm UTC](https://discuss.elastic.co/t/contenttoolongexception-when-using-reindex-api/64471/1 "2016-10-31T17:28:55Z")

</div>

Hi there!

Lately, I'm busy with finding an efficient way of reindexing a lot of data from a 1.x cluster to a 5.x cluster, and I'm evaluating some options. Right now, I'm using the Reindex API, generally it looks like a lot of success.

I'm facing issues with what it seems as responses which are too large for the target cluster to swallow.

```
[2016-10-31T18:23:35,802][DEBUG][o.e.c.RestClient] request [POST http://localhost:9205/my-index/_search?size=1000&scroll=5m&sort=_doc%3Aasc] failed
org.apache.http.ContentTooLongException: entity content is too long [15032733] for the configured buffer limit [10485760]
           at org.elasticsearch.client.HeapBufferedAsyncResponseConsumer.onEntityEnclosed(HeapBufferedAsyncResponseConsumer.java:79) ~[rest-5.0.0.jar:5.0.0]
           at org.apache.http.nio.protocol.AbstractAsyncResponseConsumer.responseReceived(AbstractAsyncResponseConsumer.java:131) ~[httpcore-nio-4.4.5.jar:4.4.5]
           at org.apache.http.impl.nio.client.MainClientExec.responseReceived(MainClientExec.java:315) ~[httpasyncclient-4.1.2.jar:4.1.2]
           at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseReceived(DefaultClientExchangeHandlerImpl.java:147) ~[httpasyncclient-4.1.2.jar:4.1.2]
           at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.responseReceived(HttpAsyncRequestExecutor.java:303) [httpcore-nio-4.4.5.jar:4.4.5]
           at org.apache.http.impl.nio.client.InternalRequestExecutor.responseReceived(InternalRequestExecutor.java:108) [httpasyncclient-4.1.2.jar:4.1.2]
           at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:255) [httpcore-nio-4.4.5.jar:4.4.5]
           at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81) [httpasyncclient-4.1.2.jar:4.1.2]
           at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39) [httpasyncclient-4.1.2.jar:4.1.2]
           at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) [httpcore-nio-4.4.5.jar:4.4.5]
           at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) [httpcore-nio-4.4.5.jar:4.4.5]
           at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) [httpcore-nio-4.4.5.jar:4.4.5]
           at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) [httpcore-nio-4.4.5.jar:4.4.5]
           at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) [httpcore-nio-4.4.5.jar:4.4.5]
           at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) [httpcore-nio-4.4.5.jar:4.4.5]
           at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588) [httpcore-nio-4.4.5.jar:4.4.5]
           at java.lang.Thread.run(Thread.java:745) [?:1.8.0_60]

```

Of course, when I reduce the scroll size the `_reindex` API uses, then it works OK more often.

Is it possible to increase this buffer with a configuration parameter? I briefly skimmed through the code, but it looks like the default value (10 MB) is overriden only in tests.

Thanks!

Haris

---

<div class="post-metadata">

**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\
**Post date:** [October 31, 2016, 5:54pm UTC](https://discuss.elastic.co/t/contenttoolongexception-when-using-reindex-api/64471/2 "2016-10-31T17:54:26Z")

</div>

> [@hariso](#):
>
> Is it possible to increase this buffer with a configuration parameter? I briefly skimmed through the code, but it looks like the default value (10 MB) is overriden only in tests.

No, it isn't possible. This was filed a few days ago but I haven't gotten to it yet:

> <https://github.com/elastic/elasticsearch/issues/21185>
>
> \<!--
> GitHub is reserved for bug reports and feature requests. The best place
> to …ask a general question is at the Elastic Discourse forums at
> https://discuss.elastic.co. If you are in fact posting a bug report or
> a feature request, please include one and only one of the below blocks
> in your new issue. Note that whether you're filing a bug report or a
> feature request, ensure that your submission is for an
> \[OS that we support\](https://www.elastic.co/support/matrix#show\_os).
> Bug reports on an OS that we do not support or feature requests
> specific to an OS that we do not support will be closed.
> \--\>
> 
> \<!--
> If you are filing a bug report, please remove the below feature
> request block and provide responses for all of the below items.
> \--\>
> 
> \*\*Elasticsearch version\*\*: 5.0
> 
> \*\*Describe the feature\*\*:
> 
> Seen a lot of error \`entity content is too long \[10800427\] for the configured buffer limit \[10485760\]\` when reindexing from remote, please make buffer limit configurable, currently it's hardcoded to 10MB.

I will get to it soon. For now you can lower the size in the way the link suggests.

---

<div class="post-metadata">

**Author:** ![hariso](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hariso/32/50668_2.png) [@hariso](https://discuss.elastic.co/u/hariso)\
**Post date:** [October 31, 2016, 8:03pm UTC](https://discuss.elastic.co/t/contenttoolongexception-when-using-reindex-api/64471/3 "2016-10-31T20:03:40Z")

</div>

Embarassing that I didn't look in GitHub! Thanks for the quick and detailed reply, and of course thanks for this great piece of software called Elasticsearch!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:08pm UTC](https://discuss.elastic.co/t/contenttoolongexception-when-using-reindex-api/64471/4 "2017-07-05T22:08:05Z")

</div>