How to avoid huge exceptional responses because of huge request

Elastic Stack Elasticsearch
1

Hi all,

we are using the Attachment plugin to extract plain text from diverse file types like PDFs. All in all, this works fine, no problems here. However, if, for some reason normally not connected to extraction as such, we get an exception as response, say an UnavailableShardsException, this response is huge and generates unnecessary network traffic, because Elasticsearch includes the complete request with its base64-encoded file content in that response!

Is it possible to configure Elasticsearch such that it leaves the request or at least the "source" part containing the base64'ed content out of the response? Or is it possible to tell it to do so per request?

Thanks and best regards
Heiko

2

You may be able to do something with logging.yml as it's log4j, but it's not really something we directly support unfortunately.

3

I'm not sure whether I understand you correctly? If a request failed the HTTP response (we're not talking about any Elasticsearch logs)) contains the original (failed) request. And in our case, this request could have been huge because it sometimes contains a base64-encoded attachment, that could well be 1GB. Could this possibly change by increasing the logging severity?

4

Right, I misunderstood. Not sure you can do anything there then.