Creating "prod" cluster with ingest nodes?


We are going to expand our test cluster with 3 "everything" nodes to split up ingest/master and data roles.
We are mostly a logcluster with winlogbeat and filebeats.

Something like this (sketch from supper table) :slight_smile:
I want to split up the filebeat recivers that gets netflow, other cisco logs from the data nodes.

Is this the right way to go, or am I totally off with this way of thinking?

(ignore the connections between ingest and data nodes) :smiley:

Regards Falk

It makes sense. The only issue with this is that if your ingest loads put too much pressure on the master(s), it can cause cluster instability.

Is it "better" to have the master roles on the data nodes?

Regards Falk

The best option is dedicated masters.

Whether data or ingest are better really comes down to your load profiles tbh.

@warkolm Thanks, I'll have that in mind.

I have recently gotten really burned with ILM in our environment, and are now going back to native with no bells and whistles :slight_smile:

Regards Falk

Ah ok, well it might be worth trying to resolve those ILM issues. It's definitely the path forward.

I'm setting up a test cluster first now :slight_smile:
So that we don't have to do EVRYTNG in prod :roll_eyes:

Our infra group gets sad, but everyone alse wins..

Regards Falk

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.