Logstash elastichsearch output in case of ES cluster

Elastic Stack Logstash
1

Hello,

I have the following situation:

  • 1 node with Logstash indexer
  • 3 nodes with Elastichsearch

How should I set the output logstash?

output {
elasticsearch {
host => "?"
cluster => "ES-cluster"
}
}

Have I to set the master hostname only or all nodes?

In the case of a single node declaration, how I can assure the HA when that node is down?

Regards.

2

Please, help me! :wink:

3

Is this Logstash 1.5? Then protocol defaults to "node". You'll probably want to use "http" instead. Assuming that from now on.

Set host to an array of hostnames. Logstash will pick a random host and move on to the next one if it's unresponsive. If you have dedicated master nodes you probably shouldn't include them in the list but otherwise list all cluster nodes. See the documentation for details.

4

Thanks for the reply. Yes, it's 1.5.

I setup the logstash output in this way:

output {
elasticsearch {
host => ["1.1.1.1","1.1.1.2","1.1.1.3"]
protocol => "http"
cluster => "ES-cluster"
}
}

The test returned this output:

Error: The setting host in plugin elasticsearch is obsolete and is no longer available. Please use the 'hosts' setting instead. You can specify multiple entries separated by comma in 'host:port' format. If you have any questions about this, you are invited to visit https://discuss.elastic.co/c/logstash and ask.

So I changed in this way:

output {
elasticsearch {
hosts => ['1.1.1.1:9200','1.1.1.2:9200','1.1.1.3:9200']
protocol => "http"
cluster => "ES-cluster"
}
}

And this is the new error:

Unknown setting 'protocol' for elasticsearch {:level=>:error}
Unknown setting 'cluster' for elasticsearch {:level=>:error}
Error: Something is wrong with your configuration.

5

As the error message indicates the protocol and cluster options no longer exist. The version of the plugin that ships with Logstash 2.0 is HTTP-only so those options aren't needed.

6

Then how we will set the cluster name in this case?

7

When using HTTP the cluster name is irrelevant.

8

Ok, Thanks magnus