Bulk indexing with java high level rest client gives error "Unable to parse response body"

Elastic Stack Elasticsearch
3

David, you were right at first point : This is the HTTP size limit.

I followed your clue "May be you can get more information by adding the httpclient to debug mode?" which is not as simple as it looks with log4j :face_with_raised_eyebrow:

I created a log4j2.xml file with this inside

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<Configuration status="DEBUG">
  <Appenders>
    <Console name="Console" target="SYSTEM_OUT">
      <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
    </Console>
  </Appenders>
  <Loggers>
    <Logger name="com.elasticsearch.client" level="DEBUG"/>
    <Logger name="org.apache.http" level="DEBUG"/>
    <Root level="error">
      <AppenderRef ref="Console"/>
    </Root>
  </Loggers>
</Configuration>

I added -Dlog4j.configurationFile=file://$SOFTWARE/conf/log4j2.xml in my command line

This gave me this in the logs

15:54:34.769 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.conn.ManagedNHttpClientConnectionImpl - http-outgoing-0 127.0.0.1:35116<->127.0.0.1:9200[ACTIVE][rw:rw]: 359 bytes read
15:54:34.770 [I/O dispatcher 1] DEBUG org.apache.http.wire - http-outgoing-0 << "HTTP/1.1 413 Request Entity Too Large[\r][\n]"
15:54:34.770 [I/O dispatcher 1] DEBUG org.apache.http.wire - http-outgoing-0 << "content-length: 0[\r][\n]"
15:54:34.770 [I/O dispatcher 1] DEBUG org.apache.http.wire - http-outgoing-0 << "[\r][\n]"
15:54:34.770 [I/O dispatcher 1] DEBUG org.apache.http.wire - http-outgoing-0 << "HTTP/1.0 400 Bad Request[\r][\n]"
15:54:34.770 [I/O dispatcher 1] DEBUG org.apache.http.wire - http-outgoing-0 << "content-type: application/json; charset=UTF-8[\r][\n]"
15:54:34.770 [I/O dispatcher 1] DEBUG org.apache.http.wire - http-outgoing-0 << "content-length: 203[\r][\n]"
15:54:34.770 [I/O dispatcher 1] DEBUG org.apache.http.wire - http-outgoing-0 << "[\r][\n]"
15:54:34.770 [I/O dispatcher 1] DEBUG org.apache.http.wire - http-outgoing-0 << "{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"text is empty (possibly HTTP/0.9)"}],"type":"illegal_argument_exception","reason":"text is empty (possibly HTTP/0.9)"},"status":400}"
15:54:34.775 [I/O dispatcher 1] DEBUG org.apache.http.headers - http-outgoing-0 << HTTP/1.1 413 Request Entity Too Large
15:54:34.775 [I/O dispatcher 1] DEBUG org.apache.http.headers - http-outgoing-0 << content-length: 0
15:54:34.775 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.client.InternalIODispatch - http-outgoing-0 [ACTIVE(299)] Response received
15:54:34.775 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.conn.ManagedNHttpClientConnectionImpl - http-outgoing-0 127.0.0.1:35116<->127.0.0.1:9200[ACTIVE][r:rw]: Event cleared [w]
15:54:34.775 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.client.MainClientExec - [exchange: 1] Response received HTTP/1.1 413 Request Entity Too Large
15:54:34.777 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.client.InternalIODispatch - http-outgoing-0 [ACTIVE(299)] Input ready
15:54:34.778 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.client.MainClientExec - [exchange: 1] Consume content
15:54:34.778 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.conn.ManagedNHttpClientConnectionImpl - http-outgoing-0 127.0.0.1:35116<->127.0.0.1:9200[ACTIVE][r:rw]: Close
15:54:34.778 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.client.InternalHttpAsyncClient - [exchange: 1] Connection cannot be kept alive
15:54:34.778 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.client.InternalHttpAsyncClient - [exchange: 1] releasing connection

A googling led me to this

https://github.com/elastic/elasticsearch/issues/2902

which made me add this to my conf/elastic.yml file and restart my node

http.max_content_length: 500mb

For good mesure, big documents means a bigger timeout

https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/_timeouts.html

And I have a working java high level REST client !

No need to bring a Xmas gift for me on your next meetup in Paris, I am so happy with this one.:hugs:

Please close the case.

Thanks David.

3 Likes