Uneven load on cluster behind ELB with RestClient

Elastic Stack Elasticsearch
1

Hello,

We've recently migrated from 1.7 to 5.6 and from Jest REST client to High-level REST Client. Our setup is 3 master nodes, 3 client nodes behind ELB and 6 data nodes in 3 availability zones in AWS. When an application starts the load is distributed evenly across 3 client nodes and 6 data nodes. But after some short period of time (10-20 minutes) I notice that client and data nodes are not loaded evenly anymore. Usually, one availability zone takes 80-85% of the load that causes two data nodes to be overloaded while 4 others do mostly nothing. Do restart, again everything is fine for some time then disbalance starts all over again. I checked java dns cache (60s) and outgoing connections from servers (there are conns for each of elb servers).

11%20AM

Has someone seen such behavior? Any ideas?

Thanks!

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.