Sane design thougts?


This is a x-post from Slack, but I think that it is perhaps easier to have a discussion here. :slight_smile:

We are setting up a new logcluster that should handle .. logs.. :slight_smile: from our infrastructure mostly. The usual suspects filebeat, winlogbeat, auditbeat etc.

Our design after some testing in docker landed on this.
But after a brief discussion in Slack I was recommended not to use "Master" role on the Ingest nodes (internal client facing).

Is this design sane, for a 14 days of hot storage and then a few months of Cold storage for "archive"?

Thoughts, pro/cons?

After some discussions in Slack this is the latest bid :slight_smile:

Removed one ingest node
And moved out the Master nodes to dedicated servers.

Regards Falk

Looking at this, a question I would have is for your ingest does that include, Elasticsearch ingest role nodes? If it does, one thing you'll need to considered is I don't believe Elasticsearch allows for layer 3 clustering, so the subnet for all servers, must be the same (I bring this up as I'm assuming mgmt-pub and mgmt-sec are different subnets). Elasticsearch Ingest nodes are becoming increasingly important with filebeat modules.


Yes, atleast that is what we had in mind.

Oh, I haven't seen that before. I thought that the cluster "talked" unicast with the other cluster nodes?
Do you have a link to the docs with this info, so I can read up about it?

Regards Falk

After looking a bit deeper, it appears that you're correct, in that if you're using unicast discovery, then you can span multiple subnets.

What I was thinking of was multi-region which is not recommended, but also not your use case. So it doesn't apply.

One last thing then, I think you'll need port 9300/tcp open between mgmt-pub and mgmt-sec so that the Elasticsearch ingest role nodes can properly talk to the rest of the ES nodes.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.