3 node cluster behind F5 VIP, how to configure?

Elastic Stack Elasticsearch
1

I have a 3 node cluster, and I'm wondering how to configure the network settings in the nodes now that an F5 VIP has entered the picture.

The hosts:
node1 - 10.1.1.1
node2 - 10.1.1.2
node3 - 10.1.1.3
The F5 VIP is set up to round robin to the above hosts.
The VIP - 10.10.10.10
I have some portal software that will be launching queries via the REST API on 9200. I want this portal to be able to make use of the round robin functionality, largely from a fault tolerance perspective.

Before the appearance of the F5 VIP, I had network.host set to 10.1.1.n, everything was working fine.

curl -X GET "10.1.1.2:9200/logstash*/_count"
{"count":3573073,"_shards":{"total":3,"successful":3,"skipped":0,"failed":0}}

Now if I try to run a query via the VIP, I get no response.

curl -X GET "10.10.10.10:9200/logstash*/_count" just hangs.

Do I need to set the VIP address as network.host so REST API is available on the VIP address, and the 10.1.1. as the network.publish_host so that cluster operations continue on 9300?

Thanks in advance.

2

Have you configured Elasticsearch to use the F5 for port 9300 traffic between the nodes? If so, that is not a good idea, nor is it supported.

If 9200 is not working, it sounds like the F5 is not doing something correctly. I can't say what as I don't know the details of how they work. Does trying to connect to each node individually still work?

3

I'd first get the cluster health via this and make sure you are green, and show all three nodes, which means they are talking, the cluster is happy, port 9300 is working, etc.
curl -X GET "10.1.1.2:9200/_cluster/health"
curl -X GET "10.1.1.2:9200/_cat/nodes?v"

If you are not green or not all nodes, you need to fix that first, making sure the nodes can see each other, connect on 9300, etc. WITHOUT the F5 being involved at all, as @warkolm nodes.

If cluster is good, then likely F5 not setup correctly and maybe thinks backends are down due to healthchecks or something, so start with checking F5 and what it thinks the pool target statuses are. And maybe iptables or other firewall issues, routing to/from the F5 (which is often on a different network, etc.)

4

No, 9300 is not subject to the F5. All 3 are on the same network, standard routing applies for cluster management.

My question is around the REST API (9200), as another host on another network will use the VIP to reach the REST API.
I ask this question because during the evolution of my Elasticsearch install :
at first I had network.host set as 127.0.0.1 and I could query using http://localhost:9200
I then updated network.host to 10.1.1.2, and could query using http://10.1.1.2:9200

And now I'm not sure what my network.host setting should be so that:
1 - The external host can query using the VIP. E.g. http://10.10.10.10:9200
2 - The cluster management traffic can use regular routing to 10.1.1.x:9300

5

Update : thanks for your help, but don't worry any more. It was a VIP name resolution problem, not an elasticsearch config problem.

1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.