413- Request too large exception

Elastic Stack Elasticsearch
1

Hi,
In our regular indexing scenario when using Bulk index API we are constantly hitting into "HTTP Status Code - 413, Exception - System.Net.WebException: The remote server returned an error: (413) Request Entity Too Large."

We do control the batch size before sending the request that is definitely lesser than 100mb. I see across posts it is recommended to increase the http.max_content_length to a greater value to address this.

Note: we are using ES5.x and NEST 2.x
In addition to the batch request that is originally being sent is there any additional payload that gets added by NEST? Or could someone suggest the best way to debug this issue and identify the root cause?

Thanks in advance!
~Divya Malini

1 Like
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

4

What do the Elasticsearch logs show at this time?

5

The ES logs file doesn't capture this exception or any detail related to the request that caused this exception. It is the InnerException message from ESResponse that has the following

ES Exception: HTTP Status Code - 413, Exception - System.Net.WebException: The remote server returned an error: (413) Request Entity Too Large.
at System.Net.HttpWebRequest.GetResponse()
at Elasticsearch.Net.HttpConnection.Request[TReturn](RequestData requestData) in C:\Repos\GitHub_elastic_elasticsearch-net\src\Elasticsearch.Net\Connection\HttpConnection.cs:line 140.

6

-> Could someone help understand what all cases ES throws 413 or content length is the only possibility?
-> Is there any ideal way through NEST that you recommend to detect the physical size of the request before being sent to ES?
-> Currently ES log doesn't capture any data regd the 413, anyway to enable it?

7

I would recommend upgrading to NEST 5.x; NEST 2.x is not compatible with Elasticsearch 5.x. Whilst it may work for the most part, it is not tested or supported against it as there are known breaking changes in Elasticsearch that are reflected in API changes in the client.

How are you currently ensuring this? One way that you could measure this to get an understanding of how big your requests are would be to set up an Action<IApiCallDetails> delegate in .OnRequestCompleted() in conjunction with .DisableDirectStreaming(), and write the request to disk, and check the size of the files.

1 Like
8

Hey Russ,
Sorry for the late reply. I will try this suggestion out and get back here. Thanks!

9

Hi Russ,
I tried this suggestion and realized that the requestbody size was exceeding the limit set with max_content_length. The difference was the jsonified request was more than the calculations we did before creating the jsonified string.

Now, is there anyway to get "apiCallDetails.ResponseBodyInBytes" value before we actually make the bulk call ?

-Divya

10

This wouldn't be possible, as the response body bytes will only be known after the request has been processed by Elasticsearch.

11

Okay, got it. Thank you!!