Where to send requests in a cluster

Elastic Stack Elasticsearch
1

We have the following cluster setup;
3 masters
2 client nodes
1 ingest node
1 data node

Each group of nodes has a dns loadbalancer infront of them.

The question is, where do I point to in my applicatons?

Do I do this only to the client requests because they have http enabled or do I create a loadbalancer infront of all the above services? Would that work?

2

Dedicated master nodes are usually used to improve resiliency and provide HA. That is however kind of pointless if you only have a single data node as that becomes a single point of failure. I would recommend either adding data nodes or simplifying your cluster topology. Just because you can have dedicated node types does not mean you should.

1 Like
3

Thanks @Christian_Dahlqvist, there are actually 3 data nodes, it's a typo. What topology would you recommend for production when every app is logging to elastic and there are a couple of http API exposed directly from elastic ?

Thanks again !

4

For most logging use cases, dedicated client nodes are necessarily required at all. Requests can usually be routed directly to the data nodes (maybe also ingest nodes). Do not send any traffic to the dedicated master nodes.

5

Can traffic be sent to the data & ingest nodes even where node.http is explicitly disabled on them?

6

Not unless you are using the transport protocol.

7

I am curious about this setting. Why is a LB in front of each group required?

8

Not really required, we are using Kubernetes and we are using a kubernetes service in front of each group. I'm open to suggestions if you think other configurations have more sense

9

I was just curious that's all. Usually the nodes communicate and distribute load amongst each other so well, that you can just chuck a LB in front of all of them.

The simplest one I have seen is

(KIbana Node) -> LB -> Subnet of (Master Nodes .... Data Nodes .... Consumer Nodes)

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.