How to forward logstash output to ES cluster and not to one host

Elastic Stack Logstash
1

Hi

I have to collect logs from around 20 servers(micro services are running there ), I have 3 elasticserach hosts, which are running in cluster formation. (2 masters and 1 data ). now when i trying to forward logs from logstatsh I do like below.

output {
elasticsearch { hosts => ["http://elasticsearchmaster:port"] }
}

but by doing above , i am only forwarding to one host, how do I forward the logs to cluster? please help.

output {
elasticsearch { hosts => ["http://jackson.cc.cec.eu.int:32406"] }
}

2

I have 3 elasticserach hosts, which are running in cluster formation. (2 masters and 1 data )

Off topic, but that doesn't sound like a good idea.

but by doing above , i am only forwarding to one host, how do I forward the logs to cluster?

The hosts option is an array so you can list multiple URLs there. You can also enable the sniffing option so that Logstash asks the cluster which nodes are available. That way Logstash will loadbalance requests to all currently eligible cluster nodes.

3

Thanks a lot, i can load balance now, but can you give me a tip on why you think its not a good idea?

4
  • You should always have an odd number of masters.
  • Having multiple masters managing a single data node doesn't make much sense. The resources are probably better spent in other ways.
  • Having masters receive indexing requests from Logstash is a bad idea.
5

It would be really helpfull if there are some guidelines on how to configure? Is there a document or something? Please help. I tried to find but no luck

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.