Where to send requests in a cluster

(Jonathan Kosgei) #1

We have the following cluster setup;
3 masters
2 client nodes
1 ingest node
1 data node

Each group of nodes has a dns loadbalancer infront of them.

The question is, where do I point to in my applicatons?

Do I do this only to the client requests because they have http enabled or do I create a loadbalancer infront of all the above services? Would that work?

(Christian Dahlqvist) #2

Dedicated master nodes are usually used to improve resiliency and provide HA. That is however kind of pointless if you only have a single data node as that becomes a single point of failure. I would recommend either adding data nodes or simplifying your cluster topology. Just because you can have dedicated node types does not mean you should.

(Gabi) #3

Thanks @Christian_Dahlqvist, there are actually 3 data nodes, it's a typo. What topology would you recommend for production when every app is logging to elastic and there are a couple of http API exposed directly from elastic ?

Thanks again !

(Christian Dahlqvist) #4

For most logging use cases, dedicated client nodes are necessarily required at all. Requests can usually be routed directly to the data nodes (maybe also ingest nodes). Do not send any traffic to the dedicated master nodes.

(Jonathan Kosgei) #5

Can traffic be sent to the data & ingest nodes even where node.http is explicitly disabled on them?

(Christian Dahlqvist) #6

Not unless you are using the transport protocol.

(Mujtaba Hussain) #7

I am curious about this setting. Why is a LB in front of each group required?

(Gabi) #8

Not really required, we are using Kubernetes and we are using a kubernetes service in front of each group. I'm open to suggestions if you think other configurations have more sense

(Mujtaba Hussain) #9

I was just curious that's all. Usually the nodes communicate and distribute load amongst each other so well, that you can just chuck a LB in front of all of them.

The simplest one I have seen is

(KIbana Node) -> LB -> Subnet of (Master Nodes .... Data Nodes .... Consumer Nodes)

(system) #10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.