HA Elastic Stack, should I use a load balancer?

Elastic Stack Elasticsearch
1

I'm building up the architecture for our Elastic Stack. We'll be ingesting tons of log data, so I'm opting for 3 dedicated master nodes and 6 dedicated data nodes.

I'm going to be using Logstash for the log ingestion and filtering, and Kibana for viewing.

My question is, should I put a load balancer in front of the 3 master nodes? Both Logstash and Kibana configurations will need an elasticsearch host address, given that I have 3 different addresses it makes sense to load balance. I'm seeing many resources online suggesting that a load balancer is unnecessary, but in the case where we can only hardcode a single access there isn't another way. Am I correct?

Furthermore, would I connect Kibana to my master node or does it need to be connected to a data node?

EDIT: I should also note that I have a 1 master, and 2 data nodes per Availability Zone.

2

If you are setting up dedicated master nodes then don't point anything at them.
The LB should point to the data nodes.

3

Not sure I understand that at all...

So both logstash and kibana will point to a load balancer that's pointing to the data nodes. What are the master nodes providing me with then?

4

Cluster availability and redundancy.