Elasitcsearch-ruby raises "Cannot get new connection from pool" error

Elastic Stack Elasticsearch
12

Hi,

I came across this trying to fix the same issue but in Logstash the fix was slightly different for me. I've added it to help anyone else who's struggling with the same problem.

:message=>"Cannot get new connection from pool.", :class=>"Elasticsearch::Transport::Transport::Error", :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/base.rb:193:in `perform_request'",

ElasticSearch would work for a bit but then stop ingesting data.

Failing configuration:

output {
  elasticsearch {
    hosts => ["https://search-*.us-east-1.es.amazonaws.com"]
    sniffing => true
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

Logstash tries to get a list of hosts from Elasticsearch but AWS's implementation has changed the format of the data returned. For more details on the specifics. https://forums.aws.amazon.com/thread.jspa?threadID=222600

Working configuration:

output
 {
  elasticsearch {
    hosts => ["https://search-*.us-east-1.es.amazonaws.com"]
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

tomwj

1 Like