Higher number of active threads with elasticsearch-java dependency

Elastic Stack Elasticsearch
21

sorry if i am asking a very basic question

  1. if we go ahead with a load balancer(LB) the client always resolves to one IP from the LB, which defeats the whole purpose of having a load balancer
  2. if we go ahead with registering the IP addresses of Elasticsearch client nodes with a sniffer, how the sniffer manages/re-establishes the connections if all the registered client nodes goes down

I understand all the approaches outlined seems reasonable but what i was looking for is some kind of best practices or best way to connect to the Elasticsearch cluster with fail safes in place.

Again sorry if i am asking something very very basic

22

You can specify a collection of IP addresses when constructing the client, rather than just a single HttpHost. Or you can use the LB as the seed address for sniffing.

23

Thank you for the information.

need one more help though.

we are seeing many connections on TIME_WAIT state and connections seems to be reset every 10 minute interval(strange behaviour).

Below is the code we are using

RestClientBuilder restClientBuilder = RestClient.builder(
                new HttpHost("host", port, "https"));

        restClientBuilder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
            @Override
            public HttpAsyncClientBuilder customizeHttpClient(final HttpAsyncClientBuilder b) {
                b.setMaxConnPerRoute(500);
                b.setMaxConnTotal(500);
                return b.setDefaultIOReactorConfig(IOReactorConfig.custom().setIoThreadCount(210).build());
            }
        });

        ElasticsearchTransport transport = new RestClientTransport(restClientBuilder.build(), new JacksonJsonpMapper());
        return new ElasticsearchClient(transport);
  1. do we have any configuration or a way to keep minimum connections similar to setMaxConnTotal(), tried searching in documentation didn't find any .
  2. Any help on why the connections will reset every 10 minute interval, does it have some internal time out set some where ?
24

Normally this kind of thing is due to some network device interfering with open connections rather than anything you can fix in software. You'll need to talk to your network infra folks about it.

25

Thank you for the quick reply.

any configuration option for a way to keep minimum connections similar to setMaxConnTotal()

26

I'm not aware of anything like that. If you're using enough connections, they'll stay open, otherwise they will get cleaned up. So maybe you're not sending enough requests?

27

Ok thank you for the information.