# Logstash complaining about no ES nodes being available

**URL:** <https://discuss.elastic.co/t/logstash-complaining-about-no-es-nodes-being-available/25939>\
**Category:** Logstash\
**Created:** [July 20, 2015, 6:48pm UTC](https://discuss.elastic.co/t/logstash-complaining-about-no-es-nodes-being-available/25939 "2015-07-20T18:48:14Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![falzm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/falzm/32/717_2.png) [@falzm](https://discuss.elastic.co/u/falzm)\
**Post date:** [July 20, 2015, 6:48pm UTC](https://discuss.elastic.co/t/logstash-complaining-about-no-es-nodes-being-available/25939/1 "2015-07-20T18:48:14Z")

</div>

Hi there

I have a weird problem with Logstash outputing to Elasticsearch: it keeps complaining in the logs about none of the configured Elasticsearch nodes being available (about once per second!), although it manages to get logs indexed anyway... Is it an awkward symptom to tell me it cannot keep up with the rate of messages to get indexed? There's no clear signs of saturation on the servers.

```auto
{:timestamp=>"2015-07-20T18:34:46.451000+0000", :message=>"Got error to send bulk of actions: None of the configured nodes are available: []", :level=>:error}
{:timestamp=>"2015-07-20T18:34:46.451000+0000", :message=>"Failed to flush outgoing items", :outgoing_count=>604, :exception=>org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available: [], :backtrace=>["org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(org/elasticsearch/client/transport/TransportClientNodesService.java:279)", "org.elasticsearch.client.transport.TransportClientNodesService.execute(org/elasticsearch/client/transport/TransportClientNodesService.java:198)", "org.elasticsearch.client.transport.support.InternalTransportClient.execute(org/elasticsearch/client/transport/support/InternalTransportClient.java:106)", "org.elasticsearch.client.support.AbstractClient.bulk(org/elasticsearch/client/support/AbstractClient.java:163)", "org.elasticsearch.client.transport.TransportClient.bulk(org/elasticsearch/client/transport/TransportClient.java:356)", "org.elasticsearch.action.bulk.BulkRequestBuilder.doExecute(org/elasticsearch/action/bulk/BulkRequestBuilder.java:164)", "org.elasticsearch.action.ActionRequestBuilder.execute(org/elasticsearch/action/ActionRequestBuilder.java:91)", "org.elasticsearch.action.ActionRequestBuilder.execute(org/elasticsearch/action/ActionRequestBuilder.java:65)", "java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:497)", "LogStash::Outputs::Elasticsearch::Protocols::NodeClient.bulk(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-1.0.1-java/lib/logstash/outputs/elasticsearch/protocol.rb:224)", "LogStash::Outputs::Elasticsearch::Protocols::NodeClient.bulk(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-1.0.1-java/lib/logstash/outputs/elasticsearch/protocol.rb:224)", "LogStash::Outputs::ElasticSearch.submit(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-1.0.1-java/lib/logstash/outputs/elasticsearch.rb:505)", "LogStash::Outputs::ElasticSearch.submit(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-1.0.1-java/lib/logstash/outputs/elasticsearch.rb:505)", "LogStash::Outputs::ElasticSearch.submit(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-1.0.1-java/lib/logstash/outputs/elasticsearch.rb:504)", "LogStash::Outputs::ElasticSearch.submit(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-1.0.1-java/lib/logstash/outputs/elasticsearch.rb:504)", "LogStash::Outputs::ElasticSearch.flush(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-1.0.1-java/lib/logstash/outputs/elasticsearch.rb:529)", "LogStash::Outputs::ElasticSearch.flush(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-1.0.1-java/lib/logstash/outputs/elasticsearch.rb:529)", "LogStash::Outputs::ElasticSearch.flush(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-1.0.1-java/lib/logstash/outputs/elasticsearch.rb:528)", "LogStash::Outputs::ElasticSearch.flush(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-1.0.1-java/lib/logstash/outputs/elasticsearch.rb:528)", "Stud::Buffer.buffer_flush(/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.20/lib/stud/buffer.rb:219)", "Stud::Buffer.buffer_flush(/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.20/lib/stud/buffer.rb:219)", "org.jruby.RubyHash.each(org/jruby/RubyHash.java:1341)", "Stud::Buffer.buffer_flush(/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.20/lib/stud/buffer.rb:216)", "Stud::Buffer.buffer_flush(/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.20/lib/stud/buffer.rb:216)", "Stud::Buffer.buffer_flush(/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.20/lib/stud/buffer.rb:193)", "Stud::Buffer.buffer_flush(/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.20/lib/stud/buffer.rb:193)", "RUBY.buffer_initialize(/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.20/lib/stud/buffer.rb:112)", "org.jruby.RubyKernel.loop(org/jruby/RubyKernel.java:1511)", "RUBY.buffer_initialize(/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.20/lib/stud/buffer.rb:110)", "java.lang.Thread.run(java/lang/Thread.java:745)"], :level=>:warn}

```

My configuration looks like this:

```auto
output {
  elasticsearch {
    protocol => "transport"
    host => ["logs1.xxxxxxx","logs2.xxxxxxx"]
    cluster => "xxxxxxx_prod_infra"
    workers => 2
  }
}

```

Setup:

- Logstash 1.5.2
- Elasticsearch 1.7.0
- Oracle Java 8u51
- GNU/Linux Debian wheezy

Thank you for your help: let me know if you need me to provide more information.

m.

---

<div class="post-metadata">

**Author:** ![falzm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/falzm/32/717_2.png) [@falzm](https://discuss.elastic.co/u/falzm)\
**Post date:** [July 31, 2015, 3:45pm UTC](https://discuss.elastic.co/t/logstash-complaining-about-no-es-nodes-being-available/25939/2 "2015-07-31T15:45:30Z")

</div>

Any help on this, please?

---

<div class="post-metadata">

**Author:** ![jordansissel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jordansissel/32/44957_2.png) [@jordansissel](https://discuss.elastic.co/u/jordansissel)\
**Post date:** [July 31, 2015, 3:55pm UTC](https://discuss.elastic.co/t/logstash-complaining-about-no-es-nodes-being-available/25939/3 "2015-07-31T15:55:08Z")

</div>

Some things to check:

- Does your cluster name match against you ES cluster's name?
- Is there a firewall/network issue blocking logstash outbound port 9300 or Elasticsearch server inbound port 9300?
- Is ES bound to port 9300?
- Do you see any other error messages? Try running Logstash with `--debug` to maybe see something else.
- Is shield enabled on your ES cluster?
- Can you try `protocol => http` and see if that works instead?

---

<div class="post-metadata">

**Author:** ![falzm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/falzm/32/717_2.png) [@falzm](https://discuss.elastic.co/u/falzm)\
**Post date:** [July 31, 2015, 4:17pm UTC](https://discuss.elastic.co/t/logstash-complaining-about-no-es-nodes-being-available/25939/4 "2015-07-31T16:17:28Z")

</div>

> [@jordansissel](#):
>
> Does your cluster name match against you ES cluster's name?

Yes.

> [@jordansissel](#):
>
> Is there a firewall/network issue blocking logstash outbound port 9300 or Elasticsearch server inbound port 9300?

No.

> [@jordansissel](#):
>
> Is ES bound to port 9300?

Yes.

> [@jordansissel](#):
>
> Is shield enabled on your ES cluster?

No, not installed.

> [@jordansissel](#):
>
> Do you see any other error messages? Try running Logstash with --debug to maybe see something else.

Tried, but my instances processes several hundreds of messages/sec it's difficult to keep up in a terminal. I haven't spotted any other error messages than the ones I've described, though.

> [@jordansissel](#):
>
> Can you try protocol =\> http and see if that works instead?

This does the trick (no more error messages in the logs), however this adds a significant network traffic overhead on my servers. Short term solution?

Thank you for your time,

m.

---

<div class="post-metadata">

**Author:** ![Tyler\_Hawkes](https://avatars.discourse-cdn.com/v4/letter/t/ee7513/32.png) [@Tyler\_Hawkes](https://discuss.elastic.co/u/Tyler_Hawkes)\
**Post date:** [August 3, 2015, 4:15pm UTC](https://discuss.elastic.co/t/logstash-complaining-about-no-es-nodes-being-available/25939/5 "2015-08-03T16:15:08Z")

</div>

I had the same issue and when I turned on debug it said that it couldn't get node status over transport. Http worked fine. You need to allow the logstash user to have access to cluster:monitor/nodes/info. Adding that to roles.yml fixed the issue for me.

---

<div class="post-metadata">

**Author:** ![falzm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/falzm/32/717_2.png) [@falzm](https://discuss.elastic.co/u/falzm)\
**Post date:** [August 4, 2015, 7:38am UTC](https://discuss.elastic.co/t/logstash-complaining-about-no-es-nodes-being-available/25939/6 "2015-08-04T07:38:55Z")

</div>

> [@Tyler\_Hawkes](#):
>
> You need to allow the logstash user to have access to cluster:monitor/nodes/info. Adding that to roles.yml fixed the issue for me.

Roles? With Shield?

---

<div class="post-metadata">

**Author:** ![Tyler\_Hawkes](https://avatars.discourse-cdn.com/v4/letter/t/ee7513/32.png) [@Tyler\_Hawkes](https://discuss.elastic.co/u/Tyler_Hawkes)\
**Post date:** [August 6, 2015, 7:41am UTC](https://discuss.elastic.co/t/logstash-complaining-about-no-es-nodes-being-available/25939/7 "2015-08-06T07:41:35Z")

</div>

Yes. I didn't see that you weren't running shield. I am and that's what fixed the issue for me.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:32am UTC](https://discuss.elastic.co/t/logstash-complaining-about-no-es-nodes-being-available/25939/8 "2017-07-06T05:32:45Z")

</div>


