# Broken Pipe errors while talking to AWS-ElsaticSearch service

**URL:** <https://discuss.elastic.co/t/broken-pipe-errors-while-talking-to-aws-elsaticsearch-service/62179>\
**Category:** Logstash\
**Created:** [October 4, 2016, 3:13pm UTC](https://discuss.elastic.co/t/broken-pipe-errors-while-talking-to-aws-elsaticsearch-service/62179 "2016-10-04T15:13:22Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![rgstreekstra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rgstreekstra/32/12292_2.png) [@rgstreekstra](https://discuss.elastic.co/u/rgstreekstra)\
**Post date:** [October 4, 2016, 3:13pm UTC](https://discuss.elastic.co/t/broken-pipe-errors-while-talking-to-aws-elsaticsearch-service/62179/1 "2016-10-04T15:13:22Z")

</div>

Hi all,

I am getting the following error:  
"INFO: I/O exception (java.net.SocketException) caught when processing request to {}-\>http://:80: Broken pipe"  
In logstash.err file during the same time in the logstash.log file I see:  
"{:timestamp=\>"2016-10-04T14:53:35.475000+0000", :message=\>"Attempted to send a bulk request to Elasticsearch configured at '["http:///"]', but Elasticsearch appears to be unreachable or down!", :error\_message=\>"Broken pipe", :class=\>"Manticore::SocketException", :level=\>:error}"

I know that the AWS service is reachable from this EC2 instance because I can do a curl to the service to get the health check of the service. Also I see indexes created in the AWS service but after a while I do not see more documents added to the index.

What can cause this error?

AWS service for ElasticSearch is version 1.5  
Logstash version 2.3.4.1  
Configuration file:  
input {  
beats {  
port =\> 5044  
congestion\_threshold =\> 180  
}  
}  
filter {  
if [beat][name] == "Raven" {  
if [type] == "analytics" {  
json {  
source =\> "message"  
}  
}  
}  
}  
output {  
elasticsearch {  
hosts =\> ":80"  
manage\_template =\> false  
index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}"  
document\_type =\> "%{[@metadata][type]}"  
flush\_size =\> 20  
workers =\> 4}  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 11, 2016, 3:48am UTC](https://discuss.elastic.co/t/broken-pipe-errors-while-talking-to-aws-elsaticsearch-service/62179/2 "2016-10-11T03:48:07Z")

</div>

> hosts =\> ":80"

Try including a hostname here? Also, are you sure it's running on port 80 rather than the usual port 9200?

---

<div class="post-metadata">

**Author:** ![rgstreekstra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rgstreekstra/32/12292_2.png) [@rgstreekstra](https://discuss.elastic.co/u/rgstreekstra)\
**Post date:** [October 11, 2016, 3:22pm UTC](https://discuss.elastic.co/t/broken-pipe-errors-while-talking-to-aws-elsaticsearch-service/62179/3 "2016-10-11T15:22:01Z")

</div>

I do have a valid hostname I scrubbed it from the script. According to AWS documentation they are using http and https for communication so that would be ports 80/443. The problem I face is that I see some document loaded into ElasticSearch and then I get broken pipe errors every so often 3 to 6 a minute. Also AWS reduces the request you can send to it to a 100Mb per request. But how do I throttle this in logstash.

Thanks for info Magnus.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:34am UTC](https://discuss.elastic.co/t/broken-pipe-errors-while-talking-to-aws-elsaticsearch-service/62179/4 "2017-07-06T04:34:42Z")

</div>