# Logstash.outputs.elasticsearch Marking url as dead. connection issue. AWS EBS datastore

**URL:** <https://discuss.elastic.co/t/logstash-outputs-elasticsearch-marking-url-as-dead-connection-issue-aws-ebs-datastore/185993>\
**Category:** Logstash\
**Created:** [June 17, 2019, 5:23am UTC](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-marking-url-as-dead-connection-issue-aws-ebs-datastore/185993 "2019-06-17T05:23:12Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Harikrishna\_Sarvaiya](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harikrishna_sarvaiya/32/36867_2.png) [@Harikrishna\_Sarvaiya](https://discuss.elastic.co/u/Harikrishna_Sarvaiya)\
**Post date:** [June 17, 2019, 5:23am UTC](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-marking-url-as-dead-connection-issue-aws-ebs-datastore/185993/1 "2019-06-17T05:23:12Z")

</div>

Checked all the existing threads couldn't help any.

[2019-06-17T02:23:50,125][WARN][logstash.outputs.elasticsearch] Marking url as dead. Last error: [LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [[https://NONE:xxxxxx@localhost:9200/](https://NONE:xxxxxx@localhost:9200/)][Manticore::SocketTimeout] Read timed out {:url=\>[https://NONE:xxxxxx@localhost:9200/](https://NONE:xxxxxx@localhost:9200/), :error\_message=\>"Elasticsearch Unreachable: [[https://NONE:xxxxxx@localhost:9200/](https://NONE:xxxxxx@localhost:9200/)][Manticore::SocketTimeout] Read timed out", :error\_class=\>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError"}  
[2019-06-17T02:23:50,126][ERROR][logstash.outputs.elasticsearch] Attempted to send a bulk request to elasticsearch' but Elasticsearch appears to be unreachable or down! {:error\_message=\>"Elasticsearch Unreachable: [[https://NONE:xxxxxx@localhost:9200/](https://NONE:xxxxxx@localhost:9200/)][Manticore::SocketTimeout] Read timed out", :class=\>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError", :will\_retry\_in\_seconds=\>2}  
[2019-06-17T02:23:51,649][WARN][logstash.outputs.elasticsearch] UNEXPECTED POOL ERROR {:e=\>#\<LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError: No Available connections\>}

Elastic components version used **5.5.2**

_There is no any error logs in elasticsearch. It is working fine._

Please help if anyone has resolve above issue.

---

<div class="post-metadata">

**Author:** ![Harikrishna\_Sarvaiya](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harikrishna_sarvaiya/32/36867_2.png) [@Harikrishna\_Sarvaiya](https://discuss.elastic.co/u/Harikrishna_Sarvaiya)\
**Post date:** [June 18, 2019, 4:12am UTC](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-marking-url-as-dead-connection-issue-aws-ebs-datastore/185993/2 "2019-06-18T04:12:29Z")

</div>

Any pointers on this ?

---

<div class="post-metadata">

**Author:** ![Harikrishna\_Sarvaiya](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harikrishna_sarvaiya/32/36867_2.png) [@Harikrishna\_Sarvaiya](https://discuss.elastic.co/u/Harikrishna_Sarvaiya)\
**Post date:** [June 21, 2019, 6:47am UTC](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-marking-url-as-dead-connection-issue-aws-ebs-datastore/185993/3 "2019-06-21T06:47:20Z")

</div>

Can someone help ! This issue is happening daily.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [June 23, 2019, 11:13am UTC](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-marking-url-as-dead-connection-issue-aws-ebs-datastore/185993/4 "2019-06-23T11:13:50Z")

</div>

It seems like you are having a problem with your Elasticsearch cluster. What is the size of this cluster? What kind of hardware does it run on? What is the output of the [cluster health API](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/cluster-health.html)?

---

<div class="post-metadata">

**Author:** ![Harikrishna\_Sarvaiya](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harikrishna_sarvaiya/32/36867_2.png) [@Harikrishna\_Sarvaiya](https://discuss.elastic.co/u/Harikrishna_Sarvaiya)\
**Post date:** [June 24, 2019, 8:31am UTC](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-marking-url-as-dead-connection-issue-aws-ebs-datastore/185993/5 "2019-06-24T08:31:54Z")

</div>

Cluster seems to be fine only in a healthy state (green status, all shards allocated correctly) .  
It is 3 node cluster. It is Amazon AWS setup with below configuration:

1. Storage : [EBS-based storage](https://aws.amazon.com/ebs/)
2. Volume type : io1
3. IOPS : 15000

---

<div class="post-metadata">

**Author:** ![Harikrishna\_Sarvaiya](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harikrishna_sarvaiya/32/36867_2.png) [@Harikrishna\_Sarvaiya](https://discuss.elastic.co/u/Harikrishna_Sarvaiya)\
**Post date:** [July 1, 2019, 1:08pm UTC](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-marking-url-as-dead-connection-issue-aws-ebs-datastore/185993/6 "2019-07-01T13:08:28Z")

</div>

Please help to solve this issue.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 1, 2019, 5:44pm UTC](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-marking-url-as-dead-connection-issue-aws-ebs-datastore/185993/7 "2019-07-01T17:44:19Z")

</div>

What instance type are you using? What is the full output of the API I asked abot earlier?

Is this by any chance related to [this thread?](https://discuss.elastic.co/t/elastic-url-dead/187068) If it is, did you change the instance type used as I recommended?

---

<div class="post-metadata">

**Author:** ![Harikrishna\_Sarvaiya](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harikrishna_sarvaiya/32/36867_2.png) [@Harikrishna\_Sarvaiya](https://discuss.elastic.co/u/Harikrishna_Sarvaiya)\
**Post date:** [July 3, 2019, 8:57am UTC](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-marking-url-as-dead-connection-issue-aws-ebs-datastore/185993/8 "2019-07-03T08:57:05Z")

</div>

Thanks,

Yes, It is somewhat related to that thread, getting the same error with same configurations.

Configurations used:

1. t2.xlarge instances
2. Size 300GiB
3. Volume type: io1 (EBS )
4. IOPS : 15,000

My further question is, I have used same above configurations on 3 different environments then why it is **working fine on one environment** and **not working in other two environments.**  
The only difference is **one environment is in "US East region"** and other **two are in "Australia region".**

**Is there anything else that I need to look into ?**

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 3, 2019, 9:00am UTC](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-marking-url-as-dead-connection-issue-aws-ebs-datastore/185993/9 "2019-07-03T09:00:01Z")

</div>

I still recommend switching instance type as I outlined in the other thread and see if this makes a difference. The fact that this works in one place and not another may well be differences in load or usage.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 31, 2019, 9:00am UTC](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-marking-url-as-dead-connection-issue-aws-ebs-datastore/185993/10 "2019-07-31T09:00:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
