Dose Low Level Rest Client not cause the same connection pressure as Transport Client

Elastic Stack Elasticsearch
1

We are using flink to write data to ES cluster.The transport client is used as the client in the flink task and the sniffer is configured.Each parallism of flink task will create a transport client, and each transport client will create 13 connections with every ES node in each cluster.This makes nodes in a large cluster to have tens of thousands of connections.Each transport client will send 13 keep alive requests to the ES node every 5s, and the sniffer will send a _cluster/state/nodes request to each node every 5s.This caused the ES node to be busy processing the requests sent by the transport client.I learned that transport client is deprecated and I consider migrating to low level rest client.
So I want to know dose Low Level Rest Client not cause the same connection pressure as Transport Client?

ES version:7.9.1

Thanks in advance.

2

The Low Level Rest client will use a lower number of connections per node.

However, the main problem is that Flink creates one client per task. It should instead use a single client shared by all tasks.

3

Can you quantify it? For example, how many connections will a Low Level Rest client create with a node, whether each connection has a scheduled keep alive task, and if so, what is the interval.We need this to evaluate.

Every Flink task is a independent Java process, so we cannot use a single client shared by all tasks.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.