Default connection timeout of Java client (actually MissingRequiredPropertyException for method /nodes/info)

Elastic Stack Elasticsearch
1

Hello!
I recently updated project dependency of project from co.elastic.clients:elasticsearch-java:7.17.22 to co.elastic.clients:elasticsearch-java:8.14.3

Everything seems working fine first hour.
After an hour exactly client not able to communicate with server.

Before I had singleton client object created via:

client = RestClient.builder(
      new HttpHost("localhost", 9200, "http"))
    ).setRequestConfigCallback(requestConfigBuilder -> {
      requestConfigBuilder.setConnectTimeout(2_000);
      requestConfigBuilder.setSocketTimeout(90_000);
      return requestConfigBuilder;
    }).build();

And it was shared between multiple threads for reading data.

In background I had daemon thread which were checking available nodes every hour

client.nodes().info().nodes().values();

In version 7 it worked fine during off hours, and connection kept alive by java client.

In version 8 seems client or something underneath is closing connection earlier. No warning messages in the logs.

I didn't find anything related in release notes. What should be changed? What timeout value? Or maybe I need to refresh somehow client session? Or client object can't be singleton and should be instantiated with every request?

2

I'd increase the frequency of your daemon thread or I'd use the sniffer.

Not sure if something changed in the low level client though.

3

I seems found reason: Enable TCP keepalives by default in Java REST clients · Issue #65213 · elastic/elasticsearch · GitHub

4

Dumb me, didn't notice incorrect level of logging exceptions.

Actually connections were working fine in my case.

This is problem of calling client.nodes().info(), it throws exception:

co.elastic.clients.transport.TransportException: node: http://127.0.0.1:9200/, status: 200, [es/nodes.info] Failed to decode response
	at co.elastic.clients.transport.ElasticsearchTransportBase.decodeTransportResponse(ElasticsearchTransportBase.java:404)
	at co.elastic.clients.transport.ElasticsearchTransportBase.getApiResponse(ElasticsearchTransportBase.java:363)
	at co.elastic.clients.transport.ElasticsearchTransportBase.performRequest(ElasticsearchTransportBase.java:147)
	at co.elastic.clients.elasticsearch.nodes.ElasticsearchNodesClient.info(ElasticsearchNodesClient.java:229)
	at com.termweb4.core.module.search.SearchModuleConfiguration$1.call(SearchModuleConfiguration.java:104)
	at com.termweb4.core.module.search.SearchModuleConfiguration$1.call(SearchModuleConfiguration.java:95)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)
Caused by: co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elasticsearch.nodes.info.NodeInfoXpack: co.elastic.clients.util.MissingRequiredPropertyException: Missing required property 'NodeInfoXpackSecurity.http' (JSON path: nodes.LnKzJ51QQlKmJ4tNcWOO5Q.settings.xpack.security) (line no=1, column no=1508, offset=-1)
	at co.elastic.clients.json.JsonpMappingException.from0(JsonpMappingException.java:134)
	at co.elastic.clients.json.JsonpMappingException.from(JsonpMappingException.java:121)
	at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:218)

I'm trying to connect to elasticsearch 7.17.22 with 8.14.3 client on my localhost with disabled xpack.security.

5

Use the same version for both client and server.

6

Yep, I was going to write that same problem occurred with 7.17.23 client too.
Method co.elastic.clients.elasticsearch.nodes.ElasticsearchNodesClient#info() of co.elastic.clients.elasticsearch.ElasticsearchClient#nodes throws this exception.

I switched to use another method for now: co.elastic.clients.elasticsearch.ElasticsearchClient#info

PS: with different server version and client I was checking this statement from documentation: Elasticsearch language clients are only backwards compatible with default distributions and without guarantees made.

7

What is the output of

mvn dependency:tree

?

8

Dependency tree is huge and I'm not allowed to disclose it.
But I'm pretty sure, here I'm using correct versions, this is what I got right now, for client:

+--- co.elastic.clients:elasticsearch-java:8.14.3
|    +--- org.elasticsearch.client:elasticsearch-rest-client:8.14.3
|    |    +--- org.apache.httpcomponents:httpclient:4.5.14
|    |    +--- org.apache.httpcomponents:httpcore:4.4.13
|    |    +--- org.apache.httpcomponents:httpasyncclient:4.1.5
|    |    +--- org.apache.httpcomponents:httpcore-nio:4.4.13
|    |    +--- commons-codec:commons-codec:1.15
|    |    \--- commons-logging:commons-logging:1.2
|    +--- com.google.code.findbugs:jsr305:3.0.2
|    +--- jakarta.json:jakarta.json-api:2.0.1 -> 2.0.2
|    \--- org.eclipse.parsson:parsson:1.0.5
|         \--- jakarta.json:jakarta.json-api:2.0.2
+--- jakarta.json:jakarta.json-api:2.0.1 -> 2.0.2

Anyway, problem happened before update to v8 client.

Before I did minor update of client from v7.17.17 to v7.17.23. And this brought problems to client.nodes().info() I just missed it because of wrong log level of application.

9

Have a look at this page. Could you try the workaround?

10

Yes, I already tried that, it bypassed first error with mapping 'NodeInfoXpackSecurity.http', but then it failed on another field, because JSON contained array value but it expected an object.

I can try later create test repo with this error.

As I said before, replacing call client.nodes.info() with client.info() solved issue, we were simply checking connectivity to cluster with this method.

Thanks for your time David.

11

Great. But could you open an issue in GitHub - elastic/elasticsearch-java: Official Elasticsearch Java Client?

I'm sure that @ltrotta will be happy to check and fix :blush:

Thanks for reporting!

12

Hello! I can confirm that client.nodes.info() is currently broken for 7.17.X versions, probably due to changes server side that were not ported to the clients. We'll fix it in newer 7.17 releases, while for 8.14.X it seems to be working correctly. Thank you for reporting this! You can still open an issue in the client repo, it helps us with tracking :slight_smile:

1 Like
13

Here we go: Client throws MissingRequiredPropertyException for method /nodes/info · Issue #855 · elastic/elasticsearch-java · GitHub

1 Like