# Dead connection to ES

**URL:** <https://discuss.elastic.co/t/dead-connection-to-es/200597>\
**Category:** Elasticsearch\
**Created:** [September 22, 2019, 4:31pm UTC](https://discuss.elastic.co/t/dead-connection-to-es/200597 "2019-09-22T16:31:17Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![sk545](https://avatars.discourse-cdn.com/v4/letter/s/7993a0/32.png) [@sk545](https://discuss.elastic.co/u/sk545)\
**Post date:** [September 22, 2019, 4:31pm UTC](https://discuss.elastic.co/t/dead-connection-to-es/200597/1 "2019-09-22T16:31:17Z")

</div>

`Preformatted text`I am trying to send some logs to Elastic search service which i have configured in AWS console , below is my sample config file  
input {  
file {  
path =\> "/var/log/logstash/\*.log"  
start\_position =\> "beginning"  
}  
}

output {  
elasticsearch {  
hosts =\> ["[https://vpc-xxxxxxxxxx.es.amazonaws.com](https://vpc-xxxxxxxxxx.es.amazonaws.com)"]  
index =\> "new-logstash-%{+YYYY.MM.dd}"  
}  
}

when i looked at logstash-plain.log i have observed below errors often

[2019-09-22T16:21:35,908][WARN][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=\>"[https://vpc-xxxx.es.amazonaws.com:9200/](https://vpc-xxxx.es.amazonaws.com:9200/)", :error\_type=\>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=\>"Elasticsearch Unreachable: [[https://xxxx.es.amazonaws.com:9200/](https://xxxx.es.amazonaws.com:9200/)][Manticore::ConnectTimeout] connect timed out"}  
which port number (443 or 9200) i should use in the output section (i have my ES in AWS as service)?

---

<div class="post-metadata">

**Author:** ![sk545](https://avatars.discourse-cdn.com/v4/letter/s/7993a0/32.png) [@sk545](https://discuss.elastic.co/u/sk545)\
**Post date:** [September 22, 2019, 8:05pm UTC](https://discuss.elastic.co/t/dead-connection-to-es/200597/2 "2019-09-22T20:05:26Z")

</div>

i have used 443 port , but i am getting below errors  
[2019-09-22T20:00:11,096][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"7.3.2"}  
[2019-09-22T20:00:12,571][INFO][org.reflections.Reflections] Reflections took 62 ms to scan 1 urls, producing 19 keys and 39 values  
[2019-09-22T20:00:13,612][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=\>{:removed=\>, :added=\>[https:xxxxx.es.amazonaws.com:443/]}}  
[2019-09-22T20:00:14,016][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=\>"https:xxxxx.es.amazonaws.com:443/"}  
[2019-09-22T20:00:14,075][INFO][logstash.outputs.elasticsearch] ES Output version determined {:es\_version=\>7}  
[2019-09-22T20:00:14,082][WARN][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document \_type {:es\_version=\>7}  
[2019-09-22T20:00:14,113][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=\>"LogStash::Outputs::ElasticSearch", :hosts=\>["https:xxxxx.es.amazonaws.com:443"]}  
[2019-09-22T20:00:14,234][WARN][org.logstash.instrument.metrics.gauge.LazyDelegatingGauge] A gauge metric of an unknown type (org.jruby.RubyArray) has been create for key: cluster\_uuids. This may result in invalid serialization. It is recommended to log an issue to the responsible developer/development team.  
[2019-09-22T20:00:14,240][INFO][logstash.javapipeline] Starting pipeline {:pipeline\_id=\>"main", "pipeline.workers"=\>2, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>50, "pipeline.max\_inflight"=\>250, :thread=\>"#\<Thread:0x5cf0790e run\>"}  
[2019-09-22T20:00:14,279][INFO][logstash.outputs.elasticsearch] Using default mapping template  
[2019-09-22T20:00:14,438][ERROR][logstash.outputs.elasticsearch] Failed to install template. {:message=\>"Got response code '401' contacting Elasticsearch at URL 'https:xxxxx.es.amazonaws.com:443/\_xpack'", :class=\>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError", :backtrace=\>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http\_client/manticore\_adapter.rb:80:in `perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:291:in`perform\_request\_to\_url'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http\_client/pool.rb:278:in `block in perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:373:in`with\_connection'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http\_client/pool.rb:277:in `perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:285:in`block in Pool'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http\_client.rb:162:in `get'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client.rb:378:in`get\_xpack\_info'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/ilm.rb:57:in `ilm_ready?'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/ilm.rb:28:in`ilm\_in\_use?'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/template\_manager.rb:14:in `install_template'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/common.rb:130:in`install\_template'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/common.rb:51:in `block in setup_after_successful_connection'"]} [2019-09-22T20:00:14,589][FATAL][logstash.runner] An unexpected error occurred! {:error=>#<LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError: LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/manticore_adapter.rb:80:in`perform\_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http\_client/pool.rb:291:in `perform_request_to_url'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:278:in`block in perform\_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http\_client/pool.rb:373:in `with_connection'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:277:in`perform\_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http\_client/pool.rb:285:in `block in Pool'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client.rb:162:in`get'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http\_client.rb:378:in `get_xpack_info'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/ilm.rb:57:in`ilm\_ready?'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/ilm.rb:28:in `ilm_in_use?'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/common.rb:52:in`block in setup\_after\_successful\_connection'"]}  
[2019-09-22T20:00:14,710][ERROR][org.logstash.Logstash] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

---

<div class="post-metadata">

**Author:** ![Sushant20](https://avatars.discourse-cdn.com/v4/letter/s/278dde/32.png) [@Sushant20](https://discuss.elastic.co/u/Sushant20)\
**Post date:** [October 16, 2019, 2:20pm UTC](https://discuss.elastic.co/t/dead-connection-to-es/200597/3 "2019-10-16T14:20:03Z")

</div>

@sk545 Were you able to resolve this issue?

---

<div class="post-metadata">

**Author:** ![sk545](https://avatars.discourse-cdn.com/v4/letter/s/7993a0/32.png) [@sk545](https://discuss.elastic.co/u/sk545)\
**Post date:** [October 17, 2019, 2:41pm UTC](https://discuss.elastic.co/t/dead-connection-to-es/200597/4 "2019-10-17T14:41:07Z")

</div>

@Sushant20, yes i did, i have used "amazon\_es" in output section and invoked our internal proxy settings  
after that , i have restarted and checked the logstash log ,all worked well

---

<div class="post-metadata">

**Author:** ![Sushant20](https://avatars.discourse-cdn.com/v4/letter/s/278dde/32.png) [@Sushant20](https://discuss.elastic.co/u/Sushant20)\
**Post date:** [October 17, 2019, 3:58pm UTC](https://discuss.elastic.co/t/dead-connection-to-es/200597/5 "2019-10-17T15:58:09Z")

</div>

Hi @sk545 Thanks for your reply. Disabling ILM in the output section worked in my case.  
Below issue was helpful: [https://github.com/elastic/logstash/issues/10722](https://github.com/elastic/logstash/issues/10722)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 14, 2019, 3:58pm UTC](https://discuss.elastic.co/t/dead-connection-to-es/200597/6 "2019-11-14T15:58:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.