My ES client always receives "Request execution exceeded the timeout"

Hi everyone,

When validating my indexes on client side, requests are sent to my ES server. After 100s, I received a timeout exception from the client.

Failure:
org.hibernate.search.util.common.SearchException: HSEARCH400034: Unable to retrieve index metadata from Elasticsearch: HSEARCH400590: Request execution exceeded the timeout of 100s, 0ms and 0ns. Request was GET /autretiers-write,autretiers-read with parameters {ignore_unavailable=true, allow_no_indices=true}
	at org.hibernate.search.backend.elasticsearch.schema.management.impl.ElasticsearchSchemaAccessor.lambda$getCurrentIndexMetadata$0(ElasticsearchSchemaAccessor.java:93)
	at org.hibernate.search.util.common.impl.Futures.lambda$handler$1(Futures.java:63)
	at java.base/java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:986)
	at java.base/java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:970)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
	at org.hibernate.search.backend.elasticsearch.client.impl.ElasticsearchClientImpl.lambda$send$2(ElasticsearchClientImpl.java:156)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.hibernate.search.util.common.SearchTimeoutException: HSEARCH400590: Request execution exceeded the timeout of 100s, 0ms and 0ns. Request was GET /autretiers-write,autretiers-read with parameters {ignore_unavailable=true, allow_no_indices=true}
	at org.hibernate.search.backend.elasticsearch.client.impl.ElasticsearchClientImpl.lambda$send$2(ElasticsearchClientImpl.java:153)
	... 6 common frames omitted
2022-07-21 16:27:22,047       o.h.s.e.r.spi.RootFailureCollector  [E] HSEARCH000521: Hibernate Search encountered a failure during bootstrap; continuing for now to list all problems, but the process will ultimately be aborted.
Context: Hibernate ORM mapping, type 'com.allegoria.notariat.business.ConservationHypotheque'

My question is, how can I speedup Elasticsearch server ? Because, my CPU is almost at zero usage and I still have free memory. My ES Server is like on idle whereas my client is waiting for too long. Something is wrong.

Note : I am all in local on windows (Client + ES Server).

Thanks.

I tried to increase request timeout to 1000s , and still timeout occurs for validating indexes. My application is not starting. If ignoring index validation, application starts. I suppose my ES server does nothing, my rest connection failed in idle state.
The logs (almost the same) :

2022-07-25 11:11:55,290      org.elasticsearch.client.RestClient  [W] request [GET http://127.0.0.1:9200/clientmoral-write,clientmoral-read?ignore_unavailable=true&allow_no_indices=true] returned 1 warnings: [299 Elasticsearch-7.16.3-4e6e4eab2297e949ec994e688dad46290d018022 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."]
2022-07-25 11:11:55,290      org.elasticsearch.client.RestClient  [D] request [GET http://127.0.0.1:9200/caisseretraite-write,caisseretraite-read?ignore_unavailable=true&allow_no_indices=true] returned [HTTP/1.1 200 OK]
2022-07-25 11:11:55,291      org.elasticsearch.client.RestClient  [W] request [GET http://127.0.0.1:9200/caisseretraite-write,caisseretraite-read?ignore_unavailable=true&allow_no_indices=true] returned 1 warnings: [299 Elasticsearch-7.16.3-4e6e4eab2297e949ec994e688dad46290d018022 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."]
2022-07-25 11:11:55,290      org.elasticsearch.client.RestClient  [W] request [GET http://127.0.0.1:9200/banque-write,banque-read?ignore_unavailable=true&allow_no_indices=true] returned 1 warnings: [299 Elasticsearch-7.16.3-4e6e4eab2297e949ec994e688dad46290d018022 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."]




2022-07-25 11:28:35,267       o.h.s.e.r.spi.RootFailureCollector  [E] HSEARCH000521: Hibernate Search encountered a failure during bootstrap; continuing for now to list all problems, but the process will ultimately be aborted.
Context: Hibernate ORM mapping, type 'com.allegoria.notariat.business.ActeParticipation'
Failure:
org.hibernate.search.util.common.SearchException: HSEARCH400034: Unable to retrieve index metadata from Elasticsearch: HSEARCH400590: Request execution exceeded the timeout of 1000s, 0ms and 0ns. Request was GET /acteparticipation-write,acteparticipation-read with parameters {ignore_unavailable=true, allow_no_indices=true}
	at org.hibernate.search.backend.elasticsearch.schema.management.impl.ElasticsearchSchemaAccessor.lambda$getCurrentIndexMetadata$0(ElasticsearchSchemaAccessor.java:93)
	at org.hibernate.search.util.common.impl.Futures.lambda$handler$1(Futures.java:63)
	at java.base/java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:986)
	at java.base/java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:970)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
	at org.hibernate.search.backend.elasticsearch.client.impl.ElasticsearchClientImpl.lambda$send$2(ElasticsearchClientImpl.java:156)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.hibernate.search.util.common.SearchTimeoutException: HSEARCH400590: Request execution exceeded the timeout of 1000s, 0ms and 0ns. Request was GET /acteparticipation-write,acteparticipation-read with parameters {ignore_unavailable=true, allow_no_indices=true}
	at org.hibernate.search.backend.elasticsearch.client.impl.ElasticsearchClientImpl.lambda$send$2(ElasticsearchClientImpl.java:153)
	... 6 common frames omitted
2022-07-25 11:28:35,271       o.h.s.e.r.spi.RootFailureCollector  [E] HSEARCH000521: Hibernate Search encountered a failure during bootstrap; continuing for now to list all problems, but the process will ultimately be aborted.
Context: Hibernate ORM mapping, type 'com.allegoria.notariat.business.AssociationSyndicale'
Failure:
org.hibernate.search.util.common.SearchException: HSEARCH400034: Unable to retrieve index metadata from Elasticsearch: HSEARCH400590: Request execution exceeded the timeout of 1000s, 0ms and 0ns. Request was GET /associationsyndicale-write,associationsyndicale-read with parameters {ignore_unavailable=true, allow_no_indices=true}
	at org.hibernate.search.backend.elasticsearch.schema.management.impl.ElasticsearchSchemaAccessor.lambda$getCurrentIndexMetadata$0(ElasticsearchSchemaAccessor.java:93)
	at org.hibernate.search.util.common.impl.Futures.lambda$handler$1(Futures.java:63)
	at java.base/java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:986)
	at java.base/java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:970)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
	at org.hibernate.search.backend.elasticsearch.client.impl.ElasticsearchClientImpl.lambda$send$2(ElasticsearchClientImpl.java:156)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.hibernate.search.util.common.SearchTimeoutException: HSEARCH400590: Request execution exceeded the timeout of 1000s, 0ms and 0ns. Request was GET /associationsyndicale-write,associationsyndicale-read with parameters {ignore_unavailable=true, allow_no_indices=true}
	at org.hibernate.search.backend.elasticsearch.client.impl.ElasticsearchClientImpl.lambda$send$2(ElasticsearchClientImpl.java:153)
	... 6 common frames omitted
2022-07-25 11:28:35,275       o.h.s.e.r.spi.RootFailureCollector  [E] HSEARCH000521: Hibernate Search encountered a failure during bootstrap; continuing for now to list all problems, but the process will ultimately be aborted.
Context: Hibernate ORM mapping, type 'com.allegoria.notariat.business.Assurance'

Is that happening for every request or from time to time?

Is your elasticsearch cluster running correctly? What are the elasticsearch logs?

Problem is solved, timeout was cause by internal framework hsearch.

Thanks.