Scroll search | Caused by: java.io.IOException: entity content is too long [226508429] for the configured buffer limit [1029760]

[vijaykriishna (Vijay Krishna)]

Hi All,

I've implemented scroll search on an index with 348,004 documents in it.

Code:

  SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder().fetchSource(includes, null)
      .size(20000);
  searchSourceBuilder.parseXContent(xContentParser);
  SearchRequest searchRequest = new SearchRequest(new String[] {index}, searchSourceBuilder).scroll(new TimeValue(60000));

I see exception as mentioned below.

Caused by: java.io.IOException: entity content is too long [226508429] for the configured buffer limit [1029760]

Caused by: org.apache.http.ConnectionClosedException: Premature end of Content-Length delimited message body (expected: 498706319; received: 343343104

Added the code:

RequestOptions.Builder builder = RequestOptions.DEFAULT.toBuilder();
builder.setHttpAsyncResponseConsumerFactory(new HttpAsyncResponseConsumerFactory.HeapBufferedResponseConsumerFactory(
    NumberUtils.toInt(1029760000));

Then i checked the behaviour by setting http.max_content_length: 2048mb in elasticsearch.yml but it was of no use.

Please help me address the issue, how to deal with this. Did i miss something ?

BR
Vijay