Unknown setting for elasticsearch

Elastic Stack Logstash
1

I'm getting this error when trying to connect logstash to my ES cluster

{:timestamp=>"2015-06-09T07:23:11.996000+0000", :message=>"Unknown setting 'aws_region' for elasticsearch", :level=>:error, :file=>"logstash/config/mixin.rb", :line=>"262", :method=>"validate_check_invalid_parameter_names"}
{:timestamp=>"2015-06-09T07:23:12.015000+0000", :message=>"Unknown setting 'discovery' for elasticsearch", :level=>:error, :file=>"logstash/config/mixin.rb", :line=>"262", :method=>"validate_check_invalid_parameter_names"}

My config:

output {
elasticsearch {
action => "index"
embedded => false
protocol => "node"
cluster => "MyESCluster"
aws_region => "eu-west-1"
discovery => "ec2"
}
}

According to the documentation both aws_region and discovery are valid settings.

Using Logstash 1.5.0 from the yum-repo on AWS Linux 2015.03.

$ yum info logstash
Loaded plugins: priorities, update-motd, upgrade-helper
Installed Packages
Name : logstash
Arch : noarch
Epoch : 1
Version : 1.5.0
Release : 1
Size : 127 M
Repo : installed
Summary : An extensible logging pipeline
URL : http://www.elasticsearch.org/overview/logstash/
License : ASL 2.0
Description : An extensible logging pipeline

Any suggestions?

2

I may have gotten a little bit further. I discovered logstash-output-elasticsearch-ec2. So I did:

$ sudo bin/plugin install logstash-output-elasticsearch-ec2

Now I get this in the logstash logs:

Jun 09, 2015 12:13:10 PM org.elasticsearch.node.internal.InternalNode
INFO: version[1.5.1], pid[10473], build[5e38401/2015-04-09T13:41:35Z]
Jun 09, 2015 12:13:10 PM org.elasticsearch.node.internal.InternalNode
INFO: initializing ...
Jun 09, 2015 12:13:10 PM org.elasticsearch.plugins.PluginsService
INFO: loaded [cloud-aws], sites []
Jun 09, 2015 12:13:18 PM org.elasticsearch.node.internal.InternalNode
INFO: initialized
Jun 09, 2015 12:13:18 PM org.elasticsearch.node.internal.InternalNode start
INFO: starting ...
Jun 09, 2015 12:13:18 PM org.elasticsearch.transport.TransportService doStart
INFO: bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/my.ip.here:9300]}
Jun 09, 2015 12:13:18 PM org.elasticsearch.discovery.DiscoveryService doStart
INFO: MyCluster/2uXwM_APR0aD4zHzxkqfYg
Jun 09, 2015 12:13:48 PM org.elasticsearch.discovery.DiscoveryService waitForInitialState
WARNING: waited for 30s and no initial state was set by the discovery
Jun 09, 2015 12:13:48 PM org.elasticsearch.node.internal.InternalNode start
INFO: started
Jun 09, 2015 12:13:53 PM org.elasticsearch.discovery.zen.ZenDiscovery joinElectedMaster
INFO: failed to send join request to master [[Owl][gQEiJjg5RMS6DwnGGNEfqQ][hostname-ESInstance1][inet[/ESInstance1.ip.here:9300]]], reason [RemoteTransportException[[Owl][inet[/ESInstance1.ip.here:9300]][internal:discovery/zen/join]]; nested: ConnectTransportException[[logstash-hostname-logstash-10473-9854][inet[/my.ip.here:9300]] connect_timeout[30s]]; nested: ConnectTimeoutException[connection timed out: /my.ip.here:9300]; ]

So it no longer complains about unknown settings. But it doesn't seem to care about the value of those settings either (discovery: ec2, but the logs says zen).

3

OK, once I checked that my logstash instance could accept incoming connections on port 9300 and not just connect to other instances on port 9300 it started working! :slight_smile: