Logstash error, Unable to connect with elasticsearch after enabling ssl

I have an elk setup (1 master ES, 3 worker es, 1 logstash, 1 kibana) with filebeat being the log collector/emitter. Post enabling x-pack and TLS, ES and Kibana is working fine. the problem is with logstash. I'm currently seeing this error in /var/log/logstash/logstash-plain.log.

[ERROR][logstash.javapipeline ][filebeat] Pipeline aborted due to error {:pipeline_id=>"filebeat", :exception=>#<Manticore::UnknownException: Unrecognized SSL message, plaintext connection?>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:37:in `block in initialize'"

I can also see below logs in the elasticsearch master server:

[2020-09-01T07:13:20,323][WARN ][o.e.x.c.s.t.n.SecurityNetty4Transport] [esmasternode1] received plaintext traffic on an encrypted channel, closing connection Netty4TcpChannel{localAddress=/10.1.1.6:9300, remoteAddress=/publicipaddress:35166} [2020-09-01T07:13:20,865][WARN ][o.e.t.TcpTransport ] [esmasternode1] exception caught on transport layer [Netty4TcpChannel{localAddress=/10.1.1.6:9300, remoteAddress=/publicipaddress:35326}], closing connection

Below are my logstash and filebeat configurations. I have setup logstash as output in filebeat and filebeat as in input in my logstash config.

Logstash.conf

input {
  beats {
    port => 5044
    ssl => true
    ssl_certificate => "/etc/logstash/logstashcert.crt"
    ssl_key => "/etc/logstash/logstashcert.key"
  }
}

filter { json { source => "message" remove_field => [ "message" ] } }

output {
  elasticsearch {
  hosts => ["https://esmasterprivateIP:9200"]
    index => "logs-%{+YYYY-MM-dd}"
    manage_template => true
    template => "/etc/logstash/conf.d/template.json"
    template_name => "mytemplate"
    ssl => true
    cacert => '/home/ubuntu/esca.pem'
    user => logstash_user
    password => mypassword

  }
}

Filebeat.conf

    output.logstash:
      workers: 2
      enabled: true
      protocol: "https"
      hosts: ['logstashprivateip:5044']
      path: "/"
      ssl:
      certificate_authorities: [“/etc/tls.crt”]

I'm unable to trace down where I'm going wrong.

Note: Filebeat is running in kubernetes, Hence the config might look slightly different as it is passed through configmap.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.