No living connections in the connection pool

Elastic Stack Logstash
1

Hi

ELK Server (Ubuntu Server 16.04)

  1. Nginx
    /usr/sbin/nginx
    LISTEN ON: 0.0.0.0:80

  2. Kibana
    /usr/share/kibana/bin/kibana --version

5.0.0-rc1

LISTEN ON: 127.0.0.1:5601

cat /etc/kibana/kibana.yml ¦ grep -v ^#

nothing

  1. Elasticsearch
    /usr/share/elasticsearch/bin/elasticsearch --version

Version: 5.0.0-rc1, Build: 13e62e1/2016-10-07T16:52:58.798Z, JVM: 1.8.0_101

LISTEN ON: 127.0.0.1:9200
LISTEN ON: 127.0.0.1:9300

cat /etc/elasticsearch/elasticsearch.yml ¦ grep -v ^#

nothing

  1. Logstash
    /usr/share/logstash/bin/logstash --version

logstash 5.0.0-rc1

LISTEN ON: :::5044
LISTEN ON: 127.0.0.1:9600

cat /etc/logstash/logstash.yml ¦ grep -v ^#

path.config: /etc/logstash/conf.d
log.level: error
path.logs: /var/log/logstash

CentOS 6.5

  1. Filebeat
    filebeat --version

filebeat version 1.3.1 (amd64)

vi /etc/filebeat/filebeat.yml

filebeat:
   prospectors:
     -
       paths:
         - /var/log/secure
         - /var/log/messages
         - /var/log/miapp/*.log
         - /var/log/*.log

       input_type: log
       document_type: syslog

   registry_file: /var/lib/filebeat/registry

 output:
   logstash:
     hosts: ["192.168.1.80:5044"]
     bulk_max_size: 1024
 #    ssl:
 #      certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]

 shipper:

 logging:
   files:
     rotateeverybytes: 10485760 # = 10MB

[2016-10-24T15:14:20,014][ERROR][logstash.outputs.elasticsearch] Attempted to send a bulk request to elasticsearch, but no there are no living connections in the connection pool. Perhaps Elasticsearch is unreachable or down? {:error_message=>"No Available connections", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError", :will_retry_in_seconds=>8}

2

In adiction, i cant configure tls/ssl, then the error showing is:

 [2016-10-24T14:54:26,019][ERROR][logstash.pipeline        ] A plugin had an unrecoverable error. Will restart this plugin.
   Plugin: <LogStash::Inputs::Beats port=>5044, ssl=>true, ssl_certificate=>"/etc/pki/tls/certs/logstash-forwarder.crt", ssl_key=>"/etc/pki/tls/private/logstash-forwarder.key", id=>"102e955a530fa0379d4140d206d5e0f79ae1d212-3", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_ae329411-d804-4785-8322-1a2bdb24a04d", enable_metric=>true, charset=>"UTF-8">, host=>"0.0.0.0", ssl_verify_mode=>"none", include_codec_tag=>true, ssl_handshake_timeout=>10000, congestion_threshold=>5, target_field_for_codec=>"message", tls_min_version=>1, tls_max_version=>1.2, cipher_suites=>["TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA38", "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256"], client_inactivity_timeout=>60>