Filebeat & Logstash SSL issue | ERR Connecting error publishing events (retrying): remote error: tls: internal error


(Franco Wolf) #1

Hi all,

I enabled SSL for filebeat and logstash and after that I recieve:

ERR Connecting error publishing events (retrying): remote error: tls: internal error

Some informations about the environment:

filebeat version 5.4.2 (amd64), libbeat 5.4.2
filebeat config

output.logstash:
hosts: ["krlv16906fblt.my-openstack.local:5044"]
#ssl.certificate_authorities: ["/etc/hdp-certs/My_Global_Root_CA.crt","/etc/hdp-certs/My_Intermediate_CA_01.crt","/etc/hdp-certs/My_Issuing_CA_01.01.crt"]
ssl_certificate_authorities => ["/etc/hdp-certs/My_Issuing_CA_01.01.crt","/etc/hdp-certs/My_Intermediate_CA_01.crt","/etc/hdp-certs/My_Global_Root_CA.crt"]
ssl.certificate: "/etc/hdp-certs/krlv16906fblt.crt"
ssl.key: "/etc/hdp-certs/hadoop_encrypted_cert.p8"
#ssl.key_passphrase:
#min_version: 1.2
#ssl.verification_mode: full
compression_level: 4
worker: 5
loadbalance: false
enabled: "True"

logstash version 5.4.2
logstash config

input{
beats{
id => 'UseCase15'
host => 'krlv16906fblt.my-openstack.local'
port => 5044
ssl => true
#ssl_certificate_authorities => ["/etc/hdp-certs/My_Global_Root_CA.crt","/etc/hdp-certs/My_Intermediate_CA_01.crt","/etc/hdp-certs/My_Issuing_CA_01.01.crt"]
ssl_certificate_authorities => ["/etc/hdp-certs/My_Issuing_CA_01.01.crt","/etc/hdp-certs/My_Intermediate_CA_01.crt","/etc/hdp-certs/My_Global_Root_CA.crt"]
ssl_certificate => "/etc/hdp-certs/krlv16906fblt.crt"
ssl_key => "/etc/hdp-certs/hadoop_encrypted_cert.p8"
#ssl_key_passphrase =>
#ssl_verify_mode => "full"
}
}

curl -v -i --cacert /etc/hdp-certs/krlv61906fblt.crt https://krlv16906fblt.my-openstack.local:5044

  • About to connect() to krlv16906fblt.my-openstack.local port 5044 (#0)
  • Trying 172.16.1.24...
  • Connected to krlv16906fblt.my-openstack.local (172.16.1.24) port 5044 (#0)
  • Initializing NSS with certpath: sql:/etc/pki/nssdb
  • CAfile: /etc/hdp-certs/krlv16906fblt.crt
    CApath: none
  • Server certificate:
  •   subject: E=webadmins@my.com,CN=krlv16906fblt.my-openstack.local,OU=ZF IT,O=MY AG,L=Town,ST=State,C=DE
    
  •   start date: Sep 27 07:03:43 2017 GMT
    
  •   expire date: Sep 26 07:03:43 2020 GMT
    
  •   common name: krlv16906fblt.my-openstack.local
    
  •   issuer: CN=My Issuing CA 01.01,OU=MY IT,O=MY
    
  • NSS error -8179 (SEC_ERROR_UNKNOWN_ISSUER)
  • Peer's Certificate issuer is not recognized.
  • Closing connection 0
    curl: (60) Peer's Certificate issuer is not recognized.
    More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA certs). If the default
bundle file isn't adequate, you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
the -k (or --insecure) option.

Logstash start without any error, if I start filebeat I recieve this issue

tail -f /var/log/filebeat/filebeat
2017-09-29T10:36:43+02:00 ERR Connecting error publishing events (retrying): remote error: tls: internal error

tail -f /var/log/logstash/logstash-plain.log
[2017-09-29T10:36:24,172][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2017-09-29T10:36:28,990][DEBUG][logstash.pipeline ] Pushing flush onto pipeline
[2017-09-29T10:36:33,989][DEBUG][logstash.pipeline ] Pushing flush onto pipeline
[2017-09-29T10:36:38,989][DEBUG][logstash.pipeline ] Pushing flush onto pipeline
[2017-09-29T10:36:43,990][DEBUG][logstash.pipeline ] Pushing flush onto pipeline
[2017-09-29T10:36:48,991][DEBUG][logstash.pipeline ] Pushing flush onto pipeline
[2017-09-29T10:36:53,991][DEBUG][logstash.pipeline ] Pushing flush onto pipeline
[2017-09-29T10:36:58,992][DEBUG][logstash.pipeline ] Pushing flush onto pipeline
[2017-09-29T10:37:03,993][DEBUG][logstash.pipeline ] Pushing flush onto pipeline
[2017-09-29T10:37:08,992][DEBUG][logstash.pipeline ] Pushing flush onto pipeline

Without SSl fielbeat and logstash are working fine. I tried a own CA instead of company CA with the same result.

Filebeat and Logstash running on the same node with same certificate.

Any ideas what is wrong with this settings?

Kind regards

Franco


(Franco Wolf) #2

additional information:

The cnf looks like that:
[req]
distinguished_name = req_distinguished_name
req_extensions = v3_req
prompt = no
[req_distinguished_name]
C = DE
ST = State
L = Town
O = My AG
OU = My IT
CN = krlv16906fblt.my-openstack.local
emailAddress = webadmins@zf.com
[v3_req]
basicConstraints = CA:FALSE
nsCertType = client, server
nsComment = "OpenSSL Generated Server Certificate"
subjectKeyIdentifier = hash
#authorityKeyIdentifier = keyid,issuer:always
extendedKeyUsage = clientAuth, serverAuth
keyUsage = keyEncipherment, dataEncipherment
subjectAltName = @alt_names
[alt_names]
#local DNS names
DNS.1 = krlv16906fblt.my-openstack.local
DNS.2 = krlv16906fblt.emea.my-world.com

Certificates was tetes with openssl, seems fine for me.


(Franco Wolf) #3

I tried different settings in filebeat and logstash config.

When I disable ssl_certificate_authorities in logstash all seems fine. Is there something missing or wrong in openssl.cnf file?


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.