# Elasticsearch Unreachable error in Logstash

**URL:** <https://discuss.elastic.co/t/elasticsearch-unreachable-error-in-logstash/75157>\
**Category:** Logstash\
**Created:** [February 15, 2017, 7:57am UTC](https://discuss.elastic.co/t/elasticsearch-unreachable-error-in-logstash/75157 "2017-02-15T07:57:09Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![elasticheart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/elasticheart/32/65189_2.png) [@elasticheart](https://discuss.elastic.co/u/elasticheart)\
**Post date:** [February 15, 2017, 7:57am UTC](https://discuss.elastic.co/t/elasticsearch-unreachable-error-in-logstash/75157/1 "2017-02-15T07:57:09Z")

</div>

Hi,

I have just upgraded from ELK 5.0.0 to 5.2.0. I have installed and set X-Pack 5.2.0 for Elasticsearch, Logstash and Kibana, and enabled monitoring alone in Kibana and Elasticsearch. Kibana and Elasticsearch are working fine. I have enabled a sample Logstash file to read from Kafka, but getting the below trace;

```
[2017-02-15T13:04:09,029][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://logstash_system:xxxxxx@localhost:9200, :path=>"/"}
[2017-02-15T13:04:09,035][WARN][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>#<URI::HTTP:0x2186e33b URL:http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s>, :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://logstash_system:xxxxxx@localhost:9200][Manticore::SocketException] Connection refused"}

```

Somebody please kind enough to tell me why is this happening and How can I fix this. Also, I want to use an IP `192.168.1.100` instead of `localhost` here. How can I do this?

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![elasticheart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/elasticheart/32/65189_2.png) [@elasticheart](https://discuss.elastic.co/u/elasticheart)\
**Post date:** [February 15, 2017, 8:20am UTC](https://discuss.elastic.co/t/elasticsearch-unreachable-error-in-logstash/75157/2 "2017-02-15T08:20:43Z")

</div>

here is the entire trace;

```
Sending Logstash's logs to data/GA5.2/logstash-5.2.0/logs which is now configured via log4j2.properties
[2017-02-15T13:44:06,720][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s]}}
[2017-02-15T13:44:06,725][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://logstash_system:xxxxxx@localhost:9200, :path=>"/"}
log4j:WARN No appenders could be found for logger (org.apache.http.client.protocol.RequestAuthCache).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
[2017-02-15T13:44:06,863][WARN][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>#<URI::HTTP:0x16168399 URL:http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s>, :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://logstash_system:xxxxxx@localhost:9200][Manticore::SocketException] Connection refused"}
[2017-02-15T13:44:06,864][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>[#<URI::HTTP:0x4e703f0d URL:http://localhost:9200>]}
[2017-02-15T13:44:06,865][INFO][logstash.pipeline] Starting pipeline {"id"=>".monitoring-logstash", "pipeline.workers"=>1, "pipeline.batch.size"=>2, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>2}
[2017-02-15T13:44:06,875][INFO][logstash.pipeline] Pipeline .monitoring-logstash started
[2017-02-15T13:44:07,008][INFO][logstash.pipeline] Starting pipeline {"id"=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>500}
[2017-02-15T13:44:07,036][INFO][logstash.pipeline] Pipeline main started
[2017-02-15T13:44:07,124][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
[2017-02-15T13:44:11,878][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://logstash_system:xxxxxx@localhost:9200, :path=>"/"}
[2017-02-15T13:44:11,889][WARN][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>#<URI::HTTP:0x721fec92 URL:http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s>, :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://logstash_system:xxxxxx@localhost:9200][Manticore::SocketException] Connection refused"}
[2017-02-15T13:44:16,545][WARN][logstash.outputs.elasticsearch] Marking url as dead. Last error: [LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s][Manticore::SocketException] Connection refused {:url=>http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s, :error_message=>"Elasticsearch Unreachable: [http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s][Manticore::SocketException] Connection refused", :error_class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError"}
[2017-02-15T13:44:16,546][ERROR][logstash.outputs.elasticsearch] Attempted to send a bulk request to elasticsearch' but Elasticsearch appears to be unreachable or down! {:error_message=>"Elasticsearch Unreachable: [http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s][Manticore::SocketException] Connection refused", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError", :will_retry_in_seconds=>2}
[2017-02-15T13:44:16,891][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://logstash_system:xxxxxx@localhost:9200, :path=>"/"}
[2017-02-15T13:44:16,896][WARN][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>#<URI::HTTP:0x604dcabe URL:http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s>, :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://logstash_system:xxxxxx@localhost:9200][Manticore::SocketException] Connection refused"}
[2017-02-15T13:44:18,550][WARN][logstash.outputs.elasticsearch] UNEXPECTED POOL ERROR {:e=>#<LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError: No Available connections>}
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 15, 2017, 8:30am UTC](https://discuss.elastic.co/t/elasticsearch-unreachable-error-in-logstash/75157/3 "2017-02-15T08:30:20Z")

</div>

> Somebody please kind enough to tell me why is this happening and How can I fix this.

Can you connect to localhost:9200 yourself? Is that where Elasticsearch is supposed to be running?

> Also, I want to use an IP 192.168.1.100 instead of localhost here. How can I do this?

Change the URL in your Logstash configuration to say 192.168.1.100 instead of localhost?

---

<div class="post-metadata">

**Author:** ![elasticheart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/elasticheart/32/65189_2.png) [@elasticheart](https://discuss.elastic.co/u/elasticheart)\
**Post date:** [February 15, 2017, 8:37am UTC](https://discuss.elastic.co/t/elasticsearch-unreachable-error-in-logstash/75157/4 "2017-02-15T08:37:34Z")

</div>

Hi @magnusbaeck

> Can you connect to localhost:9200 yourself? Is that where Elasticsearch is supposed to be running?

No, I cant. I use a remote machine with just console access and i always have to use `192.168.1.100` instead of localhost, and it works fine

> Change the URL in your Logstash configuration to say 192.168.1.100 instead of localhost?

Where is that setting? I use logstash to output just to console alone, using the setting `stdout{ codec => json }`

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discuss.elastic.co/u/jules)\
**Post date:** [February 16, 2017, 12:56pm UTC](https://discuss.elastic.co/t/elasticsearch-unreachable-error-in-logstash/75157/5 "2017-02-16T12:56:40Z")

</div>

Seeing this as well, I would love a minor update that doesn't break things...  
Another good idea is to have a single place in which to set parameters, regardless of whether init or systemd is in use, to be honest it's getting a bit pathetic, just trying to launch logstash as a different user requires changing a systemd file.

For reference, nothing at all referring to localhost in the logstash conf dir:

[root@log01 /etc/logstash] $ grep -ir localhost /etc/logstash/ .

Seeing this perpetually repeating in the logs:

[2017-02-16T14:51:18,055][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://logstash\_system:xxxxxx@localhost:9200/](http://logstash_system:xxxxxx@localhost:9200/), :path=\>"/"}  
[2017-02-16T14:51:18,059][WARN][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=\>#\<URI::HTTP:0x172c387f URL:[http://logstash\_system:xxxxxx@localhost:9200/\_xpack/monitoring/?system\_id=logstash&system\_api\_version=2&interval=1s](http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s)\>, :error\_type=\>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=\>"Elasticsearch Unreachable: [[http://logstash\_system:xxxxxx@localhost:9200/](http://logstash_system:xxxxxx@localhost:9200/)][Manticore::SocketException] Connection refused (Connection refused)"}  
[2017-02-16T14:51:23,066][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://logstash\_system:xxxxxx@localhost:9200/](http://logstash_system:xxxxxx@localhost:9200/), :path=\>"/"}  
[2017-02-16T14:51:23,071][WARN][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=\>#\<URI::HTTP:0xfdd5d4c URL:[http://logstash\_system:xxxxxx@localhost:9200/\_xpack/monitoring/?system\_id=logstash&system\_api\_version=2&interval=1s](http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s)\>, :error\_type=\>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=\>"Elasticsearch Unreachable: [[http://logstash\_system:xxxxxx@localhost:9200/](http://logstash_system:xxxxxx@localhost:9200/)][Manticore::SocketException] Connection refused (Connection refused)"}  
[2017-02-16T14:51:28,073][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://logstash\_system:xxxxxx@localhost:9200/](http://logstash_system:xxxxxx@localhost:9200/), :path=\>"/"}  
[2017-02-16T14:51:28,077][WARN][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=\>#\<URI::HTTP:0x3a615a3e URL:[http://logstash\_system:xxxxxx@localhost:9200/\_xpack/monitoring/?system\_id=logstash&system\_api\_version=2&interval=1s](http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s)\>, :error\_type=\>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=\>"Elasticsearch Unreachable: [[http://logstash\_system:xxxxxx@localhost:9200/](http://logstash_system:xxxxxx@localhost:9200/)][Manticore::SocketException] Connection refused (Connection refused)"}  
[2017-02-16T14:51:33,078][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://logstash\_system:xxxxxx@localhost:9200/](http://logstash_system:xxxxxx@localhost:9200/), :path=\>"/"}  
[2017-02-16T14:51:33,083][WARN][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=\>#\<URI::HTTP:0x56490e06 URL:[http://logstash\_system:xxxxxx@localhost:9200/\_xpack/monitoring/?system\_id=logstash&system\_api\_version=2&interval=1s](http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s)\>, :error\_type=\>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=\>"Elasticsearch Unreachable: [[http://logstash\_system:xxxxxx@localhost:9200/](http://logstash_system:xxxxxx@localhost:9200/)][Manticore::SocketException] Connection refused (Connection refused)"}  
[2017-02-16T14:51:38,084][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://logstash\_system:xxxxxx@localhost:9200/](http://logstash_system:xxxxxx@localhost:9200/), :path=\>"/"}  
[2017-02-16T14:51:38,088][WARN][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=\>#\<URI::HTTP:0x4d8ab7be URL:[http://logstash\_system:xxxxxx@localhost:9200/\_xpack/monitoring/?system\_id=logstash&system\_api\_version=2&interval=1s](http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s)\>, :error\_type=\>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=\>"Elasticsearch Unreachable: [[http://logstash\_system:xxxxxx@localhost:9200/](http://logstash_system:xxxxxx@localhost:9200/)][Manticore::SocketException] Connection refused (Connection refused)"}

Version installed:  
[root@log01 /etc/logstash] $ yum info logstash  
Loaded plugins: fastestmirror  
Loading mirror speeds from cached hostfile

- base: [mirror.wiru.co.za](http://mirror.wiru.co.za)
- epel: [fedora.is.co.za](http://fedora.is.co.za)
- extras: [mirror.wiru.co.za](http://mirror.wiru.co.za)
- updates: [mirror.wiru.co.za](http://mirror.wiru.co.za)  
Installed Packages  
Name : logstash  
Arch : noarch  
Epoch : 1  
Version : 5.2.1  
Release : 1  
Size : 167 M  
Repo : installed  
From repo : elastic-5.x  
Summary : An extensible logging pipeline  
URL : [http://www.elasticsearch.org/overview/logstash/](http://www.elasticsearch.org/overview/logstash/)  
License : ASL 2.0  
Description : An extensible logging pipeline

---

<div class="post-metadata">

**Author:** ![bradley.bristowstagg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bradley.bristowstagg/32/9954_2.png) [@bradley.bristowstagg](https://discuss.elastic.co/u/bradley.bristowstagg)\
**Post date:** [February 27, 2017, 11:16pm UTC](https://discuss.elastic.co/t/elasticsearch-unreachable-error-in-logstash/75157/7 "2017-02-27T23:16:21Z")

</div>

OK reading through the docs a bit more as I'm also hit with this issue, it seems that the initial setup NEGLECTS to mention that there are xpack settings for logstash.yml that includes the elasticsearch url for monitoring!!  
[https://www.elastic.co/guide/en/x-pack/current/monitoring-settings.html#monitoring-logstash-settings](https://www.elastic.co/guide/en/x-pack/current/monitoring-settings.html#monitoring-logstash-settings)

I'm just testing this now.

---

<div class="post-metadata">

**Author:** ![bradley.bristowstagg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bradley.bristowstagg/32/9954_2.png) [@bradley.bristowstagg](https://discuss.elastic.co/u/bradley.bristowstagg)\
**Post date:** [February 27, 2017, 11:49pm UTC](https://discuss.elastic.co/t/elasticsearch-unreachable-error-in-logstash/75157/8 "2017-02-27T23:49:28Z")

</div>

BINGO!!

```
[2017-02-28T10:19:24,022][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://10.1.1.89:9200/, :path=>"/"}
[2017-02-28T10:19:24,029][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>#<URI::HTTP:0x1a2e77f7 URL:http://10.1.1.89:9200/>}

```

You MUST add the following config lines to your /etc/logstash/logstash.yml

```
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.url: http://X.X.X.X:9200

```

Where X.X.X.X is the IP of your Elastic cluster.

EDIT: Clarity

---

<div class="post-metadata">

**Author:** ![r0bnet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/r0bnet/32/33940_2.png) [@r0bnet](https://discuss.elastic.co/u/r0bnet)\
**Post date:** [February 28, 2017, 7:36am UTC](https://discuss.elastic.co/t/elasticsearch-unreachable-error-in-logstash/75157/9 "2017-02-28T07:36:50Z")

</div>

Thanks! Now it's working fine. Maybe i'm going to deactivate the monitoring later but for now (testing purposes) it is fine!

---

<div class="post-metadata">

**Author:** ![ccampbell](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ccampbell/32/15320_2.png) [@ccampbell](https://discuss.elastic.co/u/ccampbell)\
**Post date:** [March 2, 2017, 9:15pm UTC](https://discuss.elastic.co/t/elasticsearch-unreachable-error-in-logstash/75157/10 "2017-03-02T21:15:02Z")

</div>

Thank you!

I've been sweating this problem for a few hours - it seems to have fixed logstash w/ x-pack for me as well.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 30, 2017, 9:15pm UTC](https://discuss.elastic.co/t/elasticsearch-unreachable-error-in-logstash/75157/11 "2017-03-30T21:15:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.