Tiebreaker setup

[Lafunamor]

Hi Guys

ECE automatically deploys a Tiebreaker node when ever 3 zones are available and a cluster of 2 nodes is deployed.
According to the UI the deployed Tiebreaker nodes are master eligible and are used for ingest.

Can you give me a bit more detailed description on what traffic is sent (or potentially sent) to those nodes and if (in the future) we could also disable data ingest on such nodes?
My goal is to have nodes that are only involved in management and never receive actual data but only management traffic. Is this feasible with ES and ECE?

Thx!

[Alex_Piggott]

Hi @Lafunamor

The tiebreaker nodes are simply 1GB (by default) ES instances with data disabled and master-eligible enabled, added to a 3rd zone, It is intended for HA vs management-only nodes

What I believe is closer to what you want are master eligible nodes, also possible from ECE (see cluster create/edit page)

That will create instances that are master-eligible but have data disabled, and then the data nodes are no longer eligible. This is orthogonal to tiebreakers, eg in a 2HA case you'd then have:

• zone1: 1GB master only node ("master eligible node"), N GB "data only"node
• zone2: : 1GB master only node ("master eligible node"), N GB "data only" node
• zone3: 1GB master only node ("tiebreaker")

The one limitation currently is that data requests can still be routed to both the master-eligible nodes or the data nodes. There is some work planned to allow explicit "coordinator nodes" that handle data requests but that is not expected until 1.3+

Does that answer your question?

Alex

[Lafunamor]

Hi Alex

Thanks for your reply.
I'm actually looking for a Tiebreaker node without data ingress enabled, as it should only participate in 'management functions' such as master election and shard distribution. The idea is to have such a node in a 'untrusted' environment which means that there should never be any data sent to this node.

Lafunamor

[Alex_Piggott]

@Lafunamor

Makes sense - I don't think we have proper support for this on the roadmap:

• until 1.2 placing different instances of a cluster on different hardware is not well supported
• currently the only way of restricting external traffic (which could potentially have data) from going to a given node is to put that instance into maintenance mode, but unfortunately this is problematic for your use case because applying plans can turn it off again
• you'd have to double check with the ES team that cluster nodes that have node.data: false will never receive data-related messages
• (I also don't know without more analysis if we can guarantee that credentials that would allow some level of access to "trusted instances" wouldn't leak into the "untrusted instances" ... my guess is that we wouldn't be able to guarantee that)

What's the use case for this, it seems odd to have instances that aren't trusted forming part of a trusted cluster?

Alex

[Lafunamor]

@Alex_Piggott

Thanks for your explanations.

The basic idea is to have a 3 node setup of which one node should be a a tiebreaker. The two data-bearing nodes should reside in our own DCs and the tiebreaker node could potentially be in a public cloud (or some other less trusted DC). This is just one of the multiple use cases.

Can you point someone from the ES team to this topic or should I start a new one to get some more insights regarding the feasibility of this?

Kind regards
Lafunamor

[Alex_Piggott]

Thanks for the explanation, I've heard plans similar to this. if part of threat model is a compromize of the untrusted node host then I don't believe either ES or ECE is an option, eg:

• (ES side) Someone could just edit the yaml file and turn data back on again
• (ECE side) an allocator hosting any instance of cluster X contains credentials that allows

If the threat is sniffing data, then you should be fine with 6.x which encrypts all the internal traffic

If the threat is being able to view the disk, then EAR should address that

[Lafunamor]

Hi @Alex_Piggott

Thanks for pointing this out. For my exact use-case this is not a problem. I just need a way to configure a pure voting/management node without any data sent or stored on it.

I hope this is something that will be added ad a feature in the future. I think the only missing part is the possibility to disable 'data ingress' on the tiebreaker.