Logstash fails to connect to elasticsearch with Watcher beta 2

Current Setup:
elasticsearch 1.5.1
logstash 1.5.1-1
kibana 4.0.1

After installing watcher beta 2 using the instructions on the getting started. I am unable to get Logstash to function. When I remove the license and watcher plugin Logstash begins working again. Below is the scrubbed output from Logstash failing to start.

Jun 17, 2015 1:05:27 PM org.elasticsearch.node.internal.InternalNode
INFO: [logstash-servername-25533-11314] version[1.5.1], pid[25533], build[5e38401/2015-04-09T13:41:35Z]
Jun 17, 2015 1:05:27 PM org.elasticsearch.node.internal.InternalNode
INFO: [logstash-servername-25533-11314] initializing ...
Jun 17, 2015 1:05:27 PM org.elasticsearch.plugins.PluginsService
INFO: [logstash-servername-25533-11314] loaded [], sites []
Jun 17, 2015 1:05:31 PM org.elasticsearch.node.internal.InternalNode
INFO: [logstash-servername-25533-11314] initialized
Jun 17, 2015 1:05:31 PM org.elasticsearch.node.internal.InternalNode start
INFO: [logstash-servername-25533-11314] starting ...
Jun 17, 2015 1:05:31 PM org.elasticsearch.transport.TransportService doStart
INFO: [logstash-servername-25533-11314] bound_address {inet[/0:0:0:0:0:0:0:0:9301]}, publish_address {inet[/x.x.x.x:9301]}
Jun 17, 2015 1:05:31 PM org.elasticsearch.discovery.DiscoveryService doStart
INFO: [logstash-servername-25533-11314] elasticsearch/biiIWS89TNW8YSoIcsvRrg
Jun 17, 2015 1:06:01 PM org.elasticsearch.discovery.DiscoveryService waitForInitialState
WARNING: [logstash-servername-25533-11314] waited for 30s and no initial state was set by the discovery
Jun 17, 2015 1:06:01 PM org.elasticsearch.node.internal.InternalNode start
INFO: [logstash-servername-25533-11314] started
Failed to install template: waited for [30s] {:level=>:error}

Failed to flush outgoing items {:outgoing_count=>1, :exception=>org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];[SERVICE_UNAVAILABLE/2/no master];, :backtrace=>["org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(org/elasticsearch/cluster/block/ClusterBlocks.java:151)", "org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedRaiseException(org/elasticsearch/cluster/block/ClusterBlocks.java:141)", "org.elasticsearch.action.bulk.TransportBulkAction.executeBulk(org/elasticsearch/action/bulk/TransportBulkAction.java:210)", "org.elasticsearch.action.bulk.TransportBulkAction.access$000(org/elasticsearch/action/bulk/TransportBulkAction.java:73)", "org.elasticsearch.action.bulk.TransportBulkAction$1.onFailure(org/elasticsearch/action/bulk/TransportBulkAction.java:148)", "org.elasticsearch.action.support.TransportAction$ThreadedActionListener$2.run(org/elasticsearch/action/support/TransportAction.java:137)", "java.util.concurrent.ThreadPoolExecutor.runWorker(java/util/concurrent/ThreadPoolExecutor.java:1145)", "java.util.concurrent.ThreadPoolExecutor$Worker.run(java/util/concurrent/ThreadPoolExecutor.java:615)", "java.lang.Thread.run(java/lang/Thread.java:745)"], :level=>:warn}

This issue most probably relates to a current known limitation with the elasticsearch output node protocol and watcher. see here

2 Likes

Thank you this solved my issue. After adding the protocol => "http" option to the elasticsearch output logstash is able to connect to elasticsearch with watcher installed.