Connection is closed after setting CompatibilityMode(true) of RHLC

[Andreas_Pichler]

Hello dear community,

I am currently working on making my ES7 client compatible with the ES8 cluster, but since the migration from Spring-Boot 2.3.x to 2.7.18 I am encountering the error below. To make the RestHighLevelClient used compatible with the ES8 cluster, I set the setApiCompatibilityMode(true) as described in the migration instructions.

Here are the additional classes by configuring my client.

I would be very grateful if someone could tell me a solution for this. Is the general migration of the RestClient regarding the compatibility mode correct?

package product.indexing.utils;

import org.testcontainers.elasticsearch.ElasticsearchContainer;

/**
 * docker test container for elastic search
 */
public class ElasticSearchTestContainer extends ElasticsearchContainer
{

  private static final String ELASTIC_SEARCH_DOCKER = "docker.elastic.co/elasticsearch/elasticsearch:8.11.4";

  private static final String CLUSTER_NAME = "cluster.name";

  private static final String ELASTIC_SEARCH = "elasticsearch";

  public ElasticSearchTestContainer()
  {
    super(ELASTIC_SEARCH_DOCKER);
    this.addFixedExposedPort(9200, 9200);
    this.addFixedExposedPort(9300, 9300);
    this.addEnv(CLUSTER_NAME, ELASTIC_SEARCH);
  }
}

package product.config;

import com.zaxxer.hikari.HikariConfig;
import product.indexing.IndexNameService;
import product.indexing.utils.ElasticSearchTestContainer;
import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.client.RestHighLevelClientBuilder;
import org.junit.Assert;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.testcontainers.elasticsearch.ElasticsearchContainer;

@Configuration
public class ElasticSearchClientConfig extends HikariConfig
{
  private static ElasticsearchContainer elasticsearchContainer = new ElasticSearchTestContainer();


  public static final Boolean START_ELASTIC_SEARCH = true;

  @Configuration
  protected static class ElasticSearchClientConfiguration extends HikariConfig
  {

    @Bean
    public RestClientBuilder restClientBuilder() {
      RestClientBuilder clientBuilder = RestClient.builder(HttpHost.create("localhost:9200"));
      if (START_ELASTIC_SEARCH) {
        elasticsearchContainer.start();
        System.out.println("container started: " + elasticsearchContainer.getHttpHostAddress());
        Assert.assertTrue(elasticsearchContainer.isRunning());

        final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthScope.ANY,
                new UsernamePasswordCredentials("HIDDEN", "HIDDEN"));

        clientBuilder = RestClient.builder(HttpHost.create(elasticsearchContainer.getHttpHostAddress()))
                .setHttpClientConfigCallback(http -> http.setDefaultCredentialsProvider(credentialsProvider));
      }
      return clientBuilder;
    }

    @Bean
    public RestClient restClient(RestClientBuilder restClientBuilder) {
      return restClientBuilder.build();
    }

    @Bean
    public RestHighLevelClient restHighLevelClient(RestClientBuilder restClientBuilder) {
      return new RestHighLevelClientBuilder(restClientBuilder.build())
              .setApiCompatibilityMode(true)
              .build();
    }

    @Bean
    public IndexNameService getIndexNameService() {
      return new IndexNameService("product-qa-");
    }

  }
}

Caused by: 
org.springframework.data.elasticsearch.UncategorizedElasticsearchException: java.util.concurrent.ExecutionException: org.apache.http.ConnectionClosedException: Connection is closed; nested exception is ElasticsearchExcept
ion[java.util.concurrent.ExecutionException: org.apache.http.ConnectionClosedException: Connection is closed]; nested: ExecutionException[org.apache.http.ConnectionClosedException: Connection is closed]; nested: ConnectionClosedExce
ption[Connection is closed];
Caused by: org.elasticsearch.ElasticsearchException: java.util.concurrent.ExecutionException: org.apache.http.ConnectionClosedException: Connection is closed
Caused by: java.util.concurrent.ExecutionException: org.apache.http.ConnectionClosedException: Connection is closed
Caused by: org.apache.http.ConnectionClosedException: Connection is closed

[Andreas_Pichler]

The problem was the missing Authentication for the ElasticSearchTestcontainer:

private static ElasticsearchContainer elasticsearchContainer = new ElasticSearchTestContainer().withPassword("ABC"), since my this Container is hosted by default with Authentication. Also the client requires to enable Authentication.