Logstash to Master Node or Hot Node?

Elastic Stack Elasticsearch
1

Hi,
I need some clarification in the Master, Hot Node, Warm Node design
I have setup a working cluster with a dedicated master Node thats not set to ingest any data. In my design I have a Hot-Node1 and Hot-Node2 and a Warm-Node that is configured to move the data off the hot nodes via ILM. My question is where is Logstash supposed to send the logs too? I want to have redundancy so if the Hot-Node1 where to drop then I still have logs going to Hot-Node2. Do I configure log stash to send to the Mater and let it decide even though it's just a dedicated master node and not configured to ingest any data? or do I send it to the Hot-Node1 and Hot-Node2?

Master Config

cluster.name: logging-test
Use a descriptive name for the node:
node.name: master
node.master: true
node.voting_only: false
node.ingest: false
node.ml: false
cluster.remote.connect: false

HOT-NODE1

luster.name: logging-test
node.name: hot-node
node.master: false
node.data: true
node.attr.box_type: hot

HOT-NODE2

cluster.name: logging-test
node.name: hot-node2
node.master: false
node.data: true
node.attr.box_type: hot

Logstash

output{
if "TEST" in [tags] {
elasticsearch { hosts => ["Hot-Node"]
manage_template => false
index => "logstash-%{+YYYY.MM.dd}"}
}
#stdout { codec => rubydebug }
}

Thanks

2

No, you should not send any indexing traffic to a dedicated master node. Send it to the data nodes.

Then you need three master-eligible nodes. You currently only have one, so if this node fails then your cluster will stop working.

1 Like
3

Thanks so for the logstah configuration is it as simple as this

output{
if "TEST" in [tags] {
elasticsearch { hosts => ["Hot-Node1", "Hot-Node2"]
manage_template => false
index => "logstash-%{+YYYY.MM.dd}"}
}
#stdout { codec => rubydebug }
}

Thanks

4

That looks reasonable, but I don't know Logstash very well so I can't say for sure.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.