I periodically see this in my logstash logs for various nodes:
Attempted to send a bulk request to elasticsearch' but Elasticsearch appears to be unreachable or down! {:error_message=>"Elasticsearch Unreachable: [http://server01:9200/.......
When this happens seconds later it will reestablish the connection:
Restored connection to ES instance {:url=>#<URI::HTTP:0x5d9a0942 URL:http://server01:9200/....
The nodes look fine, they don't seem to be overwhelmed looking at CPU and memory. The logstash server and the ES nodes are in the same physical location and network latency is very good. I can make curl requests to the nodes APIs fine.
What could be causing this?
Edit: Would editing the ES timeout setting resolve this?