Elastic Load Balancer as Target Host

Elastic Stack Elasticsearch
1

Hi -

I have various scenarios I would like to benchmark using different instance classes with varying node counts. It would be rather cumbersome to specify each node's IP address in the target-hosts param. I instead would like to specify the load balancer IP. I attempted to do this, but received:

[ERROR] Cannot race. Error in race control (ConnectionError(<urllib3.connection.HTTPConnection object at 0x7f4a9247a080>: Failed to establish a new connection: [Errno 111] Connection refused) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7f4a9247a080>: Failed to establish a new connection: [Errno 111] Connection refused))

Is it possible to configure either rally or load balancer to accomplish this? Of course I'm also using --pipeline=benchmark-only

If I'm forced to specify the individual node IPs, is there a limit to the parameter string length?

Thank you!

2

If the load balancer is correctly configured it should work fine. Can you share some additional details?

1 Like
3

Hello,

--target-hosts can also reference a json file, see the docs here and you can maintain this easily. For just one cluster you only need to specify the "default" key e.g.:

{
  "default": ["127.0.0.1:39201"]
}

There should be no problem with limits to the array.

re: testing behind a load balancer: as @Christian_Dahlqvist said, are you sure your load balancer works correctly? This discuss topic may also be useful to read.

Dimitris

4

Hi, thanks for the responses. I will read through the topic @dliappis, also good to know about the json option. As far as more information, it is a stack setup by a previous team which I assume is working, since it is being used. From the looks of it, the security group allows for all internal ip's and ports 80 and 9200 are open, this particular one is managing about 10 data nodes.

|Load Balancer Protocol = HTTP|Load Balancer Port = 80 |Instance Protocol = HTTP|Instance Port = 9200|Cipher = N/A |SSL Certificate = N/A|

Please let me know what other information may be useful. I also tried hitting port 80 instead of 9200, got the same results. I have no problems when hitting the individual data nodes, just the load balancer, so I'm assuming there's a configuration issue somewhere on the load balancer.

Additionally, verified I am able to perform basic curl statements to both the nodes as well as the load balancer.

5

Figured it out, needed to add a TCP listener protocol to the load balancer for rally.

6

Is there a way to have rally benchmark over HTTP instead of TCP? Would like to mimic our current application protocol and see HTTP code metrics on the EC2 side.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.