Larger than the limit

Elastic Stack Elasticsearch
1

I am trying to aad 142 MB pdf file in Elasticsearch attachment using ingest.

Error details as below

FailureReason: BadResponse while attempting PUT on http://127.0.0.1:6379/index-stress-test-10oct2018-r1/document/b6091cc6-c2b7-40c3-9f26-7187223d3cfb?pipeline=attachments

Audit trail of this API call:

OriginalException: Elasticsearch.Net.ElasticsearchClientException: Request failed to execute. Call: Status code 503 from: PUT /index-stress-test-10oct2018-r1/document/b6091cc6-c2b7-40c3-9f26-7187223d3cfb?pipeline=attachments. ServerError: Type: circuit_breaking_exception Reason: "[parent] Data too large, data for [<http_request>] would be [363305103/346.4mb], which is larger than the limit of [363285708/346.4mb]"

Request:

<Request stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>

Response:

<Response stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>

Exception:

Elasticsearch.Net.ElasticsearchClientException: Request failed to execute. Call: Status code 503 from: PUT /index-stress-test-10oct2018-r1/document/b6091cc6-c2b7-40c3-9f26-7187223d3cfb?pipeline=attachments. ServerError: Type: circuit_breaking_exception Reason: "[parent] Data too large, data for [<http_request>] would be [363305103/346.4mb], which is larger than the limit of [363285708/346.4mb]"

Could you please let me know fix for same.

2

Hi Harshal,

The parent circuit breaker is calculated by adding the estimated usage of each of the child circuit breakers, so the first thing I would do is to check the /_nodes/stats/breaker output to see what each of the child breakers' estimated field is. Then you can see what is using most of the circuit breaker memory.

The second thing is that the parent circuit breaker defaults to 70% of the HEAP, and with a 346mb limit that means you're using a ~500mb heap (assuming you haven't changed the limit), I don't recommend trying to index a 142mb pdf file into Elasticsearch with such a small heap, keep in mind that at least 142mb will have to be used Just for networking buffers, no including the memory used for analysis, lucene, etc

3

Thanks.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.