your patch seems to fix the issue.
The time to add a remote connection now stays under one second.
I think I might have found the problem for
it: Transport Client: Adding more nodes causes more scheduled reconnect tasks · Issue #1062 · elastic/elasticsearch · GitHub. Backported
to 0.16 branch, can you give it a go and see if it fixes it?
On Friday, June 24, 2011 at 3:25 PM, Michel Conrad wrote:
I just run the connection test again, and tried connecting to 20 nodes.
The time in the debug log is always for adding a single connection to the
pool.
INFO - initializing connections to search servers
14:17:14,942 11/06/24 DEBUG elasticsearch.threadpool:70 - [Topaz]
creating thread_pool [cached], type [cached], keep_alive [30s]
DEBUG - [Topaz] creating thread_pool [cached], type [cached], keep_alive
[30s]
14:17:14,943 11/06/24 DEBUG elasticsearch.threadpool:70 - [Topaz]
creating thread_pool [index], type [cached], keep_alive [5m]
DEBUG - [Topaz] creating thread_pool [index], type [cached], keep_alive [5m]
14:17:14,943 11/06/24 DEBUG elasticsearch.threadpool:70 - [Topaz]
creating thread_pool [search], type [cached], keep_alive [5m]
DEBUG - [Topaz] creating thread_pool [search], type [cached], keep_alive
[5m]
14:17:14,944 11/06/24 DEBUG elasticsearch.threadpool:70 - [Topaz]
creating thread_pool [percolate], type [cached], keep_alive [5m]
DEBUG - [Topaz] creating thread_pool [percolate], type [cached], keep_alive
[5m]
14:17:14,944 11/06/24 DEBUG elasticsearch.threadpool:70 - [Topaz]
creating thread_pool [management], type [scaling], min [1], size [20],
keep_alive [30s]
DEBUG - [Topaz] creating thread_pool [management], type [scaling], min
[1], size [20], keep_alive [30s]
14:17:14,947 11/06/24 DEBUG elasticsearch.threadpool:70 - [Topaz]
creating thread_pool [merge], type [scaling], min [1], size [20],
keep_alive [30s]
DEBUG - [Topaz] creating thread_pool [merge], type [scaling], min [1],
size [20], keep_alive [30s]
14:17:14,947 11/06/24 DEBUG elasticsearch.threadpool:70 - [Topaz]
creating thread_pool [snapshot], type [scaling], min [1], size [20],
keep_alive [5m]
DEBUG - [Topaz] creating thread_pool [snapshot], type [scaling], min
[1], size [20], keep_alive [5m]
14:17:14,967 11/06/24 DEBUG client.transport:70 - [Topaz]
node_sampler_interval[5s]
DEBUG - [Topaz] node_sampler_interval[5s]
14:17:14,985 11/06/24 DEBUG nio.NioProviderMetadata:70 - Using the
autodetected NIO constraint level: 0
DEBUG - Using the autodetected NIO constraint level: 0
14:17:15,085 11/06/24 DEBUG transport.netty:70 - [Topaz] Connected to
node [[#transport#-1][inet[192.168.0.154:9300]]]
DEBUG - [Topaz] Connected to node
[[#transport#-1][inet[192.168.0.154:9300]]]
DEBUG - time: 175ms
14:17:15,235 11/06/24 DEBUG transport.netty:70 - [Topaz] Connected to
node [[#transport#-2][inet[192.168.0.220:9300]]]
DEBUG - [Topaz] Connected to node
[[#transport#-2][inet[192.168.0.220:9300]]]
DEBUG - time: 114ms
14:17:15,441 11/06/24 DEBUG transport.netty:70 - [Topaz] Connected to
node [[#transport#-3][inet[192.168.0.204:9300]]]
DEBUG - [Topaz] Connected to node
[[#transport#-3][inet[192.168.0.204:9300]]]
DEBUG - time: 204ms
14:17:15,698 11/06/24 DEBUG transport.netty:70 - [Topaz] Connected to
node [[#transport#-4][inet[192.168.0.216:9300]]]
DEBUG - [Topaz] Connected to node
[[#transport#-4][inet[192.168.0.216:9300]]]
DEBUG - time: 257ms
14:17:16,043 11/06/24 DEBUG transport.netty:70 - [Topaz] Connected to
node [[#transport#-5][inet[192.168.0.213:9300]]]
DEBUG - [Topaz] Connected to node
[[#transport#-5][inet[192.168.0.213:9300]]]
DEBUG - time: 343ms
14:17:16,371 11/06/24 DEBUG transport.netty:70 - [Topaz] Connected to
node [[#transport#-6][inet[192.168.0.147:9300]]]
DEBUG - [Topaz] Connected to node
[[#transport#-6][inet[192.168.0.147:9300]]]
DEBUG - time: 328ms
14:17:16,802 11/06/24 DEBUG transport.netty:70 - [Topaz] Connected to
node [[#transport#-7][inet[192.168.0.150:9300]]]
DEBUG - [Topaz] Connected to node
[[#transport#-7][inet[192.168.0.150:9300]]]
DEBUG - time: 432ms
14:17:17,225 11/06/24 DEBUG transport.netty:70 - [Topaz] Connected to
node [[#transport#-8][inet[192.168.0.149:9300]]]
DEBUG - [Topaz] Connected to node
[[#transport#-8][inet[192.168.0.149:9300]]]
DEBUG - time: 423ms
14:17:17,696 11/06/24 DEBUG transport.netty:70 - [Topaz] Connected to
node [[#transport#-9][inet[192.168.0.144:9300]]]
DEBUG - [Topaz] Connected to node
[[#transport#-9][inet[192.168.0.144:9300]]]
DEBUG - time: 472ms
14:17:18,241 11/06/24 DEBUG transport.netty:70 - [Topaz] Connected to
node [[#transport#-10][inet[192.168.0.140:9300]]]
DEBUG - [Topaz] Connected to node
[[#transport#-10][inet[192.168.0.140:9300]]]
DEBUG - time: 545ms
14:17:18,763 11/06/24 DEBUG transport.netty:70 - [Topaz] Connected to
node [[#transport#-11][inet[192.168.0.157:9300]]]
DEBUG - [Topaz] Connected to node
[[#transport#-11][inet[192.168.0.157:9300]]]
DEBUG - time: 521ms
14:17:19,382 11/06/24 DEBUG transport.netty:70 - [Topaz] Connected to
node [[#transport#-12][inet[192.168.0.155:9300]]]
DEBUG - [Topaz] Connected to node
[[#transport#-12][inet[192.168.0.155:9300]]]
DEBUG - time: 620ms
14:17:19,997 11/06/24 DEBUG transport.netty:70 - [Topaz] Connected to
node [[#transport#-13][inet[192.168.0.207:9300]]]
DEBUG - [Topaz] Connected to node
[[#transport#-13][inet[192.168.0.207:9300]]]
DEBUG - time: 613ms
14:17:21,235 11/06/24 DEBUG transport.netty:70 - [Topaz] Connected to
node [[#transport#-14][inet[192.168.0.202:9300]]]
DEBUG - [Topaz] Connected to node
[[#transport#-14][inet[192.168.0.202:9300]]]
DEBUG - time: 3138ms
14:17:24,466 11/06/24 DEBUG transport.netty:70 - [Topaz] Connected to
node [[#transport#-15][inet[192.168.0.156:9300]]]
DEBUG - [Topaz] Connected to node
[[#transport#-15][inet[192.168.0.156:9300]]]
DEBUG - time: 9853ms
14:17:34,400 11/06/24 DEBUG transport.netty:70 - [Topaz] Connected to
node [[#transport#-16][inet[192.168.0.205:9300]]]
DEBUG - [Topaz] Connected to node
[[#transport#-16][inet[192.168.0.205:9300]]]
DEBUG - time: 13659ms
14:17:48,143 11/06/24 DEBUG transport.netty:70 - [Topaz] Connected to
node [[#transport#-17][inet[192.168.0.221:9300]]]
DEBUG - [Topaz] Connected to node
[[#transport#-17][inet[192.168.0.221:9300]]]
DEBUG - time: 16698ms
14:18:04,938 11/06/24 DEBUG transport.netty:70 - [Topaz] Connected to
node [[#transport#-18][inet[192.168.0.211:9300]]]
DEBUG - [Topaz] Connected to node
[[#transport#-18][inet[192.168.0.211:9300]]]
DEBUG - time: 19951ms
14:18:24,988 11/06/24 DEBUG transport.netty:70 - [Topaz] Connected to
node [[#transport#-19][inet[192.168.0.217:9300]]]
DEBUG - [Topaz] Connected to node
[[#transport#-19][inet[192.168.0.217:9300]]]
DEBUG - time: 23470ms
14:18:48,536 11/06/24 DEBUG transport.netty:70 - [Topaz] Connected to
node [[#transport#-20][inet[192.168.0.143:9300]]]
DEBUG - [Topaz] Connected to node
[[#transport#-20][inet[192.168.0.143:9300]]]
DEBUG - time: 26651ms
On Thu, Jun 23, 2011 at 1:49 PM, Shay Banon
shay.banon@elasticsearch.com wrote:
There shouldn't be, when adding another address, it will go and try to
connect to it, while still being connected to the rest of the servers.
On Tuesday, June 21, 2011 at 12:54 PM, Michel Conrad wrote:
Hi,
in order to handle many searchrequests in a short time, I have been
threading the requests. After some request
have been fired, the sending of new requests slows down and some
requests are getting timeouts.
I am using searchRequestBuilder.execute().actionGet(60000);
As I have been suspecting to number of connections per hosts, I have
also been trying to increase
the number of servers my application is connecting to. I am currently using
TransportClient c = new
TransportClient(ImmutableSettings.settingsBuilder().put("cluster.name",
cluster_name));
for (String h : hosts) {
c.addTransportAddress(new InetSocketTransportAddress(h, port)); }
If the number of hosts gets bigger than 10, the time for
addTransportAddress increases exponentially.
Is there any synchronization code that is unable to handle more
connections or are there some internal limitations?
Regards,
Michel