# Logstash 2.2.0 \> elasticsearch output connectivity issue

**URL:** <https://discuss.elastic.co/t/logstash-2-2-0-elasticsearch-output-connectivity-issue/41534>\
**Category:** Logstash\
**Created:** [February 11, 2016, 5:59pm UTC](https://discuss.elastic.co/t/logstash-2-2-0-elasticsearch-output-connectivity-issue/41534 "2016-02-11T17:59:59Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sijo\_C](https://avatars.discourse-cdn.com/v4/letter/s/9f8e36/32.png) [@Sijo\_C](https://discuss.elastic.co/u/Sijo_C)\
**Post date:** [February 11, 2016, 5:59pm UTC](https://discuss.elastic.co/t/logstash-2-2-0-elasticsearch-output-connectivity-issue/41534/1 "2016-02-11T17:59:59Z")

</div>

I have filebeat,logstash 2.2.0, ES 2.1.0 working in dev.

Strange connectivity issue when moving to prod:

- ES cluster behind an elb (aws vpc), no TLS
- "curl prod-elb-xxx:9200"  
works from my logstash box (in datacenter), returns "cluster\_name" etc  
I can do curl -XPUT a new doc, as well
- On logstash startup, it can't connect to ES & pipeline stalls
- Since curl works, I can't imagine any other issue with acl or securityGroups
- config & error log below.

Any Clues, appreciated !

**output config**  
output {

```
    elasticsearch {
        hosts => ["prod-elb-xxx:9200"]
        sniffing => true
        manage_template => false
        index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
        document_type => "%{[@metadata][type]}"
    }
}

```

**Error in log**  
{:timestamp=\>"2016-02-10T16:02:56.814000-0500", :message=\>"Attempted to send a bulk request to Elasticsearch configured at '["[http://prod-elb-xxx:9200/](http://prod-elb-xxx:9200/)"]', but an error occurred and it failed! Are you sure you can reach elasticsearch from this machine using the configuration provided?", :client\_config=\>{:hosts=\>["[http://prod-elb-xxx:9200/](http://prod-elb-xxx:9200/)"], :ssl=\>nil, :transport\_options=\>{:socket\_timeout=\>0, :request\_timeout=\>0, :proxy=\>nil, :ssl=\>{}}, :transport\_class=\>Elasticsearch::Transport::Transport::HTTP::Manticore, :logger=\>nil, :tracer=\>nil, :reload\_connections=\>false, :retry\_on\_failure=\>false, :reload\_on\_failure=\>false, :randomize\_hosts=\>false}, :error\_message=\>"connect timed out", :error\_class=\>"Manticore::ConnectTimeout", :backtrace=\>["/opt/logstash/vendor/bundle/jruby/1.9/gems/manticore-0.5.2-java/lib/manticore/response.rb:37:in `initialize'", "org/jruby/RubyProc.java:281:in`call'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/manticore-0.5.2-java/lib/manticore/response.rb:79:in `call'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/manticore-0.5.2-java/lib/manticore/response.rb:256:in ..... java/lib/logstash/pipeline.rb:206:in`worker\_loop'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0-java/lib/logstash/pipeline.rb:175:in `start\_workers'"], :level=\>:error}

```
{:timestamp=>"2016-02-10T16:02:56.820000-0500", :message=>"connect timed out", :class=>"Manticore::ConnectTimeout", :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/manticore-0.5.2-java/lib/manticore/response.rb:37:in `initialize'", "org/jruby/RubyProc.java:281:in `call'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/manticore-0.5.2-java/lib/manticore/response.rb:79:in `call'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/manticore-0.5.2-java/lib/manticore/response.rb:256:in `call_once'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/manticore-0.5.2-java/lib/manticore/response.rb:153:in `code'", ... "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.4.1-java/lib/logstash/outputs/elasticsearch/common.rb:84:in `retrying_submit'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.4.1-java/lib/logstash/outputs/elasticsearch/common.rb:28:in `multi_receive'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0-java/lib/logstash/output_delegator.rb:119:in `worker_multi_receive'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0-java/lib/logstash/output_delegator.rb:65:in `multi_receive'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0-java/lib/logstash/pipeline.rb:275:in `output_batch'", "org/jruby/RubyHash.java:1342:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0-java/lib/logstash/pipeline.rb:275:in `output_batch'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0-java/lib/logstash/pipeline.rb:206:in `worker_loop'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0-java/lib/logstash/pipeline.rb:175:in `start_workers'"], :level=>:warn}
```

---

<div class="post-metadata">

**Author:** ![james.bromberger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james.bromberger/32/7744_2.png) [@james.bromberger](https://discuss.elastic.co/u/james.bromberger)\
**Post date:** [February 12, 2016, 2:15am UTC](https://discuss.elastic.co/t/logstash-2-2-0-elasticsearch-output-connectivity-issue/41534/2 "2016-02-12T02:15:01Z")

</div>

Similar issue here with LogStash 2.2.0, against admittedly older ElasticSearch 1.7.1 - but behind an AWS ELB, with telnet/curl working fine to ElasticSearch (and no issues seen like this with Logstash 1.5.2 or 1.5.4 for months:

{:timestamp=\>"2016-02-11T08:04:03.580000+0000", :message=\>"elasticsearch-logsink.u\_xxxxx\_.com:9200 failed to respond", :class=\>"Manticore::ClientProtocolException", :backtrace=\>["/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/manticore-0.5.2-java/lib/manticore/response.rb:37:in `initialize'", "org/jruby/RubyProc.java:281:in`call'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/manticore-0.5.2-java/lib/manticore/response.rb:79:in `call'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/manticore-0.5.2-java/lib/manticore/response.rb:256:in`call\_once'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/manticore-0.5.2-java/lib/manticore/response.rb:153:in `code'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/http/manticore.rb:71:in`perform\_request'", "org/jruby/RubyProc.java:281:in `call'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/base.rb:201:in`perform\_request'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/http/manticore.rb:54:in `perform_request'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/client.rb:125:in`perform\_request'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/elasticsearch-api-1.0.15/lib/elasticsearch/api/actions/bulk.rb:87:in `bulk'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.4.1-java/lib/logstash/outputs/elasticsearch/http_client.rb:53:in`non\_threadsafe\_bulk'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.4.1-java/lib/logstash/outputs/elasticsearch/http\_client.rb:38:in `bulk'", "org/jruby/ext/thread/Mutex.java:149:in`synchronize'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.4.1-java/lib/logstash/outputs/elasticsearch/http\_client.rb:38:in `bulk'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.4.1-java/lib/logstash/outputs/elasticsearch/common.rb:160:in`safe\_bulk'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.4.1-java/lib/logstash/outputs/elasticsearch/common.rb:99:in `submit'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.4.1-java/lib/logstash/outputs/elasticsearch/common.rb:84:in`retrying\_submit'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.4.1-java/lib/logstash/outputs/elasticsearch/common.rb:28:in `multi_receive'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0-java/lib/logstash/output_delegator.rb:119:in`worker\_multi\_receive'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0-java/lib/logstash/output\_delegator.rb:118:in `worker_multi_receive'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0-java/lib/logstash/output_delegator.rb:65:in`multi\_receive'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0-java/lib/logstash/pipeline.rb:275:in `output_batch'", "org/jruby/RubyHash.java:1342:in`each'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0-java/lib/logstash/pipeline.rb:275:in `output_batch'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0-java/lib/logstash/pipeline.rb:206:in`worker\_loop'", "/usr/local/NTR/logstash-2.2.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0-java/lib/logstash/pipeline.rb:175:in `start\_workers'"], :level=\>:warn}

This is running in a Docker 1.8 container, and after this message, the container exists (exit code 137).

---

<div class="post-metadata">

**Author:** ![Sijo\_C](https://avatars.discourse-cdn.com/v4/letter/s/9f8e36/32.png) [@Sijo\_C](https://discuss.elastic.co/u/Sijo_C)\
**Post date:** [February 12, 2016, 2:59am UTC](https://discuss.elastic.co/t/logstash-2-2-0-elasticsearch-output-connectivity-issue/41534/3 "2016-02-12T02:59:02Z")

</div>

Also, if I use filebeat's elasticsearch output directly , skipping logstash, to the same url for elasticsearch server , it is working. I am ruling out network issue.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [February 12, 2016, 6:12am UTC](https://discuss.elastic.co/t/logstash-2-2-0-elasticsearch-output-connectivity-issue/41534/4 "2016-02-12T06:12:54Z")

</div>

> [@Sijo\_C](#):
>
> sniffing =\> true

If you are accessing Elasticsearch through a loed balancer, you will not be able to connect directly to the nodes, so sniffing should not be enabled.

---

<div class="post-metadata">

**Author:** ![Sijo\_C](https://avatars.discourse-cdn.com/v4/letter/s/9f8e36/32.png) [@Sijo\_C](https://discuss.elastic.co/u/Sijo_C)\
**Post date:** [February 12, 2016, 3:53pm UTC](https://discuss.elastic.co/t/logstash-2-2-0-elasticsearch-output-connectivity-issue/41534/5 "2016-02-12T15:53:50Z")

</div>

That's it. Thanks a bunch. Wish error in log implied that.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:11am UTC](https://discuss.elastic.co/t/logstash-2-2-0-elasticsearch-output-connectivity-issue/41534/6 "2017-07-06T05:11:46Z")

</div>