# Frequent Creation of Threads by elastic.clients

**URL:** <https://discuss.elastic.co/t/frequent-creation-of-threads-by-elastic-clients/362505>\
**Category:** Elasticsearch\
**Tags:** language-clients\
**Created:** [July 4, 2024, 2:50am UTC](https://discuss.elastic.co/t/frequent-creation-of-threads-by-elastic-clients/362505 "2024-07-04T02:50:06Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![tom\_ding](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tom_ding/32/134298_2.png) [@tom\_ding](https://discuss.elastic.co/u/tom_ding)\
**Post date:** [July 4, 2024, 2:50am UTC](https://discuss.elastic.co/t/frequent-creation-of-threads-by-elastic-clients/362505/1 "2024-07-04T02:50:06Z")

</div>

I am using Elasticsearch Java API Client [8.13] and Elasticsearch (8.13), but I encounter the following exception:

复制代码

```auto
unable to create native thread: possibly out of memory or process/resource limits reached
    at java.base/java.lang.Thread.start0(Native Method)
    at java.base/java.lang.Thread.start(Thread.java:809)
    at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase.start(CloseableHttpAsyncClientBase.java:83)
    at org.elasticsearch.client.RestClientBuilder.build(RestClientBuilder.java:302)
    at kepler.schedulesearcher.dao.FixQueryTeacherByEsDao.<init>(FixQueryTeacherByEsDao.java:75)
    at kepler.schedulesearcher.dao.FixQueryTeacherByEsDao$FastClassByGuice$49598721.GUICE$TRAMPOLINE(<generated>)
    at kepler.schedulesearcher.dao.FixQueryTeacherByEsDao$FastClassByGuice$49598721.apply(<generated>)
    at com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:82)
    at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114)
    at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
    at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:296)
    at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:40)
    at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:60)
    at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:113)
    at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
    at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:296)
    at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:40)
    at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:60)
    at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:113)
    at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
    at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:296)
    at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1100)
     ... 38 common frames omitted

```

I use VisualVM to monitor my project, and I found many threads similar to `elasticsearch-rest-client-550-thread-2`.

When I restart the project, I notice that every request adds two groups of similar threads, each group having 9 threads.

I have tried creating a singleton ElasticSearchClient object in the project to share this object, but every time I use this object, such threads are still created.

Occasionally, in concurrent situations, CPU usage reaches 100%. I am not sure if the memory overflow is related to this phenomenon, but I want to confirm whether it is normal for the `ElasticSearch-rest-client-()- thread-()` threads to be created frequently.

my code：

```auto
@Singleton
public class ElasticsearchClientManagerUtils {
    final static Logger logger
            = LoggerFactory.getLogger(ElasticSearchConnectUtils.class);
    private static ElasticsearchClientManagerUtils instance;
    private static RestClient restClient;
    private static ElasticsearchClient client;
    private static ElasticsearchTransport transport;
    private ElasticsearchClientManagerUtils() {
        try {
            String elasticSearchHost ="xxx.xxx.xxx.xxx";
            String elasticSearchPort = "9200";
            String elasticSearchSchema = "https";
            String elasticSearchUser = "elastic";
            String elasticSearchPassword = " **************";
            String caCertificate = " *********************";
            String credentials = elasticSearchUser + ":" + elasticSearchPassword;
            String esAuthorization = Base64.getEncoder().encodeToString(credentials.getBytes());
            RestClientBuilder builder = RestClient.builder(
                    new HttpHost(elasticSearchHost, Integer.parseInt(elasticSearchPort), elasticSearchSchema)).setHttpClientConfigCallback(httpAsyncClientBuilder -> {
                httpAsyncClientBuilder.setMaxConnTotal(2);
                httpAsyncClientBuilder.setMaxConnPerRoute(2); 
                return httpAsyncClientBuilder;
            });
            Header[] defaultHeaders =
                    new Header[]{new BasicHeader("Authorization",
                            "Basic "+esAuthorization)};
            SSLContext sslContext = TransportUtils
                    .sslContextFromCaFingerprint(caCertificate);
            BasicCredentialsProvider credsProv = new BasicCredentialsProvider();
            credsProv.setCredentials(
                    AuthScope.ANY, new UsernamePasswordCredentials(elasticSearchUser, elasticSearchPassword)
            );
            builder.setDefaultHeaders(defaultHeaders);
            builder.setHttpClientConfigCallback(hc->hc.setSSLContext(sslContext).setDefaultCredentialsProvider(credsProv));
            restClient = builder.build();
            // Create the transport with a Jackson mapper
            transport = new RestClientTransport(
                    restClient, new JacksonJsonpMapper());
            client = new ElasticsearchClient(transport);
        } catch (Exception e) {
            logger.info("create esClient error {}",e.getMessage());
        }
    }
    public static synchronized ElasticsearchClientManagerUtils getInstance() {
        if (instance == null) {
            instance = new ElasticsearchClientManagerUtils();
        }
        logger.info(instance.toString());
        return instance;
    }

    public ElasticsearchClient getClient() {
        return client;
    }

    public void close() {
        try {
            if (restClient != null) {
                restClient.close();
                transport.close();
            }
        } catch (Exception e) {
            logger.info("close es client error:{}",e.getMessage());
        }
    }
}

```

**Monitoring content** :

 ![Snipaste_10-49-22-217-217](https://us1.discourse-cdn.com/elastic/original/3X/c/8/c86e534d506c92a240ed0348b558ca3a92107299.png)

---

<div class="post-metadata">

**Author:** ![tom\_ding](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tom_ding/32/134298_2.png) [@tom\_ding](https://discuss.elastic.co/u/tom_ding)\
**Post date:** [July 4, 2024, 5:19am UTC](https://discuss.elastic.co/t/frequent-creation-of-threads-by-elastic-clients/362505/2 "2024-07-04T05:19:05Z")

</div>

From #Elastic Search to #Elasticsearch

---

<div class="post-metadata">

**Author:** ![tom\_ding](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tom_ding/32/134298_2.png) [@tom\_ding](https://discuss.elastic.co/u/tom_ding)\
**Post date:** [July 4, 2024, 5:19am UTC](https://discuss.elastic.co/t/frequent-creation-of-threads-by-elastic-clients/362505/3 "2024-07-04T05:19:06Z")

</div>

Added #language-clients

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [July 4, 2024, 5:50am UTC](https://discuss.elastic.co/t/frequent-creation-of-threads-by-elastic-clients/362505/4 "2024-07-04T05:50:55Z")

</div>

> [@tom\_ding](#):
>
> `elasticsearch-rest-client-550-thread-2`.

This name indicates that you have created 550 separate REST client instances. You should have only one.

> [@tom\_ding](#):
>
> ```auto
> unable to create native thread: possibly out of memory or process/resource limits reached
> at java.base/java.lang.Thread.start0(Native Method)
> at java.base/java.lang.Thread.start(Thread.java:809)
> at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase.start(CloseableHttpAsyncClientBase.java:83)
> at org.elasticsearch.client.RestClientBuilder.build(RestClientBuilder.java:302)
> at kepler.schedulesearcher.dao.FixQueryTeacherByEsDao.<init>(FixQueryTeacherByEsDao.java:75)
> 
> ```

This stack trace indicates you're calling `RestClientBuilder#build` to create a new client instance each time you construct a new `FixQueryTeacherByEsDao` instance. That sounds very wrong to me.

---

<div class="post-metadata">

**Author:** ![anantguptadbl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anantguptadbl/32/134637_2.png) [@anantguptadbl](https://discuss.elastic.co/u/anantguptadbl)\
**Post date:** [July 4, 2024, 6:09am UTC](https://discuss.elastic.co/t/frequent-creation-of-threads-by-elastic-clients/362505/5 "2024-07-04T06:09:31Z")

</div>

@tom_ding can you apply some loggers in your ` private ElasticsearchClientManagerUtils()` so that we are sure that for any reason this snippet is not getting triggered more than once

```auto
RestClientBuilder builder = RestClient.builder(
                    new HttpHost(elasticSearchHost, Integer.parseInt(elasticSearchPort), elasticSearchSchema)).setHttpClientConfigCallback(httpAsyncClientBuilder -> {
                httpAsyncClientBuilder.setMaxConnTotal(2);
                httpAsyncClientBuilder.setMaxConnPerRoute(2); 
                return httpAsyncClientBuilder;
            });

```

Also are you running multiple instance of your java code on K8 etc?

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [July 4, 2024, 6:28am UTC](https://discuss.elastic.co/t/frequent-creation-of-threads-by-elastic-clients/362505/6 "2024-07-04T06:28:25Z")

</div>

> [@anantguptadbl](#):
>
> can you apply some loggers in your ` private ElasticsearchClientManagerUtils()`

The stack trace shows that it's not that.

> [@anantguptadbl](#):
>
> Also are you running multiple instance of your java code on K8 etc?

If so, they'd be different processes, but the OP shows there are hundreds of client instances within a single process. So it's not that either.

---

<div class="post-metadata">

**Author:** ![tom\_ding](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tom_ding/32/134298_2.png) [@tom\_ding](https://discuss.elastic.co/u/tom_ding)\
**Post date:** [July 4, 2024, 7:19am UTC](https://discuss.elastic.co/t/frequent-creation-of-threads-by-elastic-clients/362505/7 "2024-07-04T07:19:51Z")

</div>

Thank you for your reply.

Yes, the stack trace information is not from my provided code.

The code I am using creates an instance for each request and then releases the resources. It looks something like this:

```auto
RestClientBuilder builder = RestClient.builder(
                new HttpHost(EsHost, Integer.parseInt(EsPort), EsScheme));
Header[] defaultHeaders =
                new Header[]{new BasicHeader("Authorization",
                        "Basic "+EsAuthorization)};
SSLContext sslContext = TransportUtils
                .sslContextFromCaFingerprint(caCertificate);
BasicCredentialsProvider credsProv = new BasicCredentialsProvider();
credsProv.setCredentials(
                AuthScope.ANY, new UsernamePasswordCredentials(EsUserName, EsUserPassword)
);
builder.setDefaultHeaders(defaultHeaders);
builder.setHttpClientConfigCallback(hc->hc.setSSLContext(sslContext).setDefaultCredentialsProvider(credsProv));
RestClient restClient = builder.build();
// Create the transport with a Jackson mapper
ElasticsearchTransport transport = new RestClientTransport(
                restClient, new JacksonJsonpMapper());
// And create the API client
ElasticsearchClient esClient = new ElasticsearchClient(transport);
....
....
....
restClient.close();

```

The code I initially provided was an attempt to create a singleton ElasticsearchClient object to see if it would still repeatedly create `elasticsearch-rest-client-%d-thread-%d` threads. The result is that it still creates a lot of threads.

---

<div class="post-metadata">

**Author:** ![tom\_ding](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tom_ding/32/134298_2.png) [@tom\_ding](https://discuss.elastic.co/u/tom_ding)\
**Post date:** [July 4, 2024, 7:53am UTC](https://discuss.elastic.co/t/frequent-creation-of-threads-by-elastic-clients/362505/8 "2024-07-04T07:53:11Z")

</div>

```auto
RestClientBuilder builder = RestClient.builder(
                    new HttpHost(elasticSearchHost, Integer.parseInt(elasticSearchPort), elasticSearchSchema)).setHttpClientConfigCallback(httpAsyncClientBuilder -> {
                httpAsyncClientBuilder.setMaxConnTotal(2);
                httpAsyncClientBuilder.setMaxConnPerRoute(2); 
logger.info("create RestClientBuilder");
                return httpAsyncClientBuilder;
            });

ElasticsearchClient client = ElasticsearchClientManagerUtils.getInstance().getClient();
logger.info("Get ElasticsearchClient "+client.toString());

```

I have used logging:

```auto
15:39:11.283 [qtp210281271-21] INFO o.h.validator.internal.util.Version - HV000001: Hibernate Validator 4.3.1.Final
15:39:11.435 [qtp210281271-21] INFO k.s.utils.ElasticSearchConnectUtils - kepler.schedulesearcher.utils.ElasticsearchClientManagerUtils@7c2160fe
15:39:11.435 [qtp210281271-21] INFO k.s.dao.QueryTeacherByEsDao - Get ElasticsearchClient co.elastic.clients.elasticsearch.ElasticsearchClient@11d3370a
15:39:36.517 [qtp210281271-15] INFO k.s.utils.ElasticSearchConnectUtils - kepler.schedulesearcher.utils.ElasticsearchClientManagerUtils@7c2160fe
15:39:36.517 [qtp210281271-15] INFO k.s.dao.QueryTeacherByEsDao - Get ElasticsearchClient co.elastic.clients.elasticsearch.ElasticsearchClient@11d3370a

```

more threads will still be created

---

<div class="post-metadata">

**Author:** ![tom\_ding](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tom_ding/32/134298_2.png) [@tom\_ding](https://discuss.elastic.co/u/tom_ding)\
**Post date:** [July 4, 2024, 7:53am UTC](https://discuss.elastic.co/t/frequent-creation-of-threads-by-elastic-clients/362505/9 "2024-07-04T07:53:49Z")

</div>

![Snipaste_15-45-58-144-144](https://us1.discourse-cdn.com/elastic/original/3X/0/9/09495173d0aff94c25b2c00268407062ec8e56dd.png)

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [July 4, 2024, 7:56am UTC](https://discuss.elastic.co/t/frequent-creation-of-threads-by-elastic-clients/362505/10 "2024-07-04T07:56:07Z")

</div>

You're still creating loads of separate client instances. Create one, and reuse it for every request.

---

<div class="post-metadata">

**Author:** ![tom\_ding](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tom_ding/32/134298_2.png) [@tom\_ding](https://discuss.elastic.co/u/tom_ding)\
**Post date:** [July 4, 2024, 8:00am UTC](https://discuss.elastic.co/t/frequent-creation-of-threads-by-elastic-clients/362505/11 "2024-07-04T08:00:16Z")

</div>

Thank you for pointing out the cause of the problem. I will fix it in the program.

---

<div class="post-metadata">

**Author:** ![anantguptadbl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anantguptadbl/32/134637_2.png) [@anantguptadbl](https://discuss.elastic.co/u/anantguptadbl)\
**Post date:** [July 4, 2024, 10:35am UTC](https://discuss.elastic.co/t/frequent-creation-of-threads-by-elastic-clients/362505/12 "2024-07-04T10:35:05Z")

</div>

If you are using Spring Boot, you can leverage the `ElasticsearchConfiguration` class with the `spring-boot-starter-data-elasticsearch` library and override the `clientConfiguration`

---

<div class="post-metadata">

**Author:** ![tom\_ding](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tom_ding/32/134298_2.png) [@tom\_ding](https://discuss.elastic.co/u/tom_ding)\
**Post date:** [July 5, 2024, 1:41am UTC](https://discuss.elastic.co/t/frequent-creation-of-threads-by-elastic-clients/362505/13 "2024-07-05T01:41:50Z")

</div>

Thanks for your response, I'm not using Spring Boot. My application did create a large number of client instances, which I have now solved.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 2, 2024, 1:41am UTC](https://discuss.elastic.co/t/frequent-creation-of-threads-by-elastic-clients/362505/14 "2024-08-02T01:41:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.