Output to unicast ES Cluster, how?

Elastic Stack Logstash
1

I have a 3 node ES cluster setup and im trying to figure out how to make Logstash output to a cluster. I know that I can add something similar to the following:

elasticsearch {
                cluster => "logstash-cluster"
                host => "logstash"
 }

But if I understand things correctly this will only work with a multicast configuration. My ES cluster is configured in a unicast configuration. So I would like to not send to only one host incase that host is down for some reason. How do I configure Logstash?

2

It's not unicast or multicast on their own. It's unicast and multicast discovery. What this means for ES is that hosts provided for unicast discovery are only helpers. Each node gets the entire cluster state after finding even one member.

That said, you shouldn't be able to even use the cluster option with the 2.x releases of Logstash in the elasticsearch output block. It defaults to use the http protocol now, which is preferred. The client will route the documents to the cluster for you.

3

Ok so I modified my output as follows:

output {
  elasticsearch { hosts => ["192.168.20.20:9200", "192.168.20.21:9200", "192.168.20.22:9200"] }
}

Now what happens if Logstash is trying to send logs to 192.168.20.20, but its offline? Will it fail-over to 192.168.20.21?

4

Yep, and it uses all 3 in load balancing.