"Failed to derive xcontent" on REST bulk with ES 5.0

Hi all,
I'm migrating code from 2.4 to 5.0 and I'm facing an issue. The code bellow (that uses RestClient v 5.0.0) fails to execute the whole bulk and raises a

java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:294)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.elasticsearch.client.ResponseException: POST http://127.0.0.1:9200/beam/test/_bulk?refresh=true: HTTP/1.1 400 Bad Request
{"error":{"root_cause":[{"type":"parse_exception","reason":"Failed to derive xcontent"}],"type":"parse_exception","reason":"Failed to derive xcontent"},"status":400}
	at org.elasticsearch.client.RestClient$1.completed(RestClient.java:310)
	at org.elasticsearch.client.RestClient$1.completed(RestClient.java:299)
	at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:119)
	at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:177)
	at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436)
	at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326)
	at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
	at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
	at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
	at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
	at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
	at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)

But this code works well with ES v2.4. The request json payload seems to be correct because it is executed properly by sense.

POST /beam/test/_bulk?refresh=true
{ "index" : {} }
{"scientist":"Einstein", "id":0}
{ "index" : {} }
{"scientist":"Darwin", "id":1}

I did not see any breaking change in the website about this. What am I doing wrong?

  /** Inserts the given number of test documents into Elasticsearch. */
  static void insertTestDocuments(String index, String type, long numDocs, RestClient restClient)
      throws IOException {
    List<String> data =
        ElasticSearchIOTestUtils.createDocuments(
            numDocs, ElasticSearchIOTestUtils.InjectionMode.DO_NOT_INJECT_INVALID_DOCS);
    StringBuilder bulkRequest = new StringBuilder();
    for (String document : data) {
      bulkRequest.append(String.format("{ \"index\" : {} }%n%s%n", document));
    }
    String endPoint = String.format("/%s/%s/_bulk", index, type);
    HttpEntity requestBody =
        new NStringEntity(bulkRequest.toString(), ContentType.APPLICATION_JSON);
    Response response = restClient.performRequest("POST", endPoint,
        Collections.singletonMap("refresh", "true"), requestBody,
        new BasicHeader("", ""));
    JsonNode searchResult = ElasticsearchIO.parseResponse(response);
    boolean errors = searchResult.path("errors").asBoolean();
    if (errors){
      throw new IOException(
          String.format("Failed to insert test documents in index %s", index));
    }
  }


  /**
   * Generates a list of test documents for insertion.
   *
   * @param numDocs Number of docs to generate
   * @param injectionMode {@link InjectionMode} that specifies whether to insert malformed documents
   * @return the list of json String representing the documents
   */
  static List<String> createDocuments(long numDocs, InjectionMode injectionMode) {
    String[] scientists = {
      "Einstein",
      "Darwin",
      "Copernicus",
      "Pasteur",
      "Curie",
      "Faraday",
      "Newton",
      "Bohr",
      "Galilei",
      "Maxwell"
    };
    ArrayList<String> data = new ArrayList<>();
    for (int i = 0; i < numDocs; i++) {
      int index = i % scientists.length;
      // insert 2 malformed documents
      if (InjectionMode.INJECT_SOME_INVALID_DOCS.equals(injectionMode) && (i == 6 || i == 7)) {
        data.add(String.format("{\"scientist\";\"%s\", \"id\":%d}", scientists[index], i));
      } else {
        data.add(String.format("{\"scientist\":\"%s\", \"id\":%d}", scientists[index], i));
      }
    }
    return data;
  }

The call to restClient#performRequest should not contain new BasicHeader("", ""). If headers are empty you don't have to provide a fake one and you can directly call:

HttpEntity requestBody =
        new NStringEntity(bulkRequest.toString(), ContentType.APPLICATION_JSON);
    Response response = restClient.performRequest("POST", endPoint,
        Collections.singletonMap("refresh", "true"), requestBody);

In 5.4 when you set an empty header it fails with the following message:
empty headers are not allowed []

.. and it works when the header is removed.

Thanks Jim! Indeed it works on both ES versions when removing the empty headers.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.