TLS connection failed because of certificate signed by unknown authority

[caoping]

Hello,
I have spent two days in configuring filebeat TLS, and always encountered below error. Can anyone give me some tips on how to resolve this issue?

The error message in filebeat side:

Below is the steps and configuration.

1. I generated a self-signed certificate, named ca.crt.

openssl genrsa -out ca/ca-key.pem 1024
openssl req -new -out ca/ca.csr -key ca/ca.key -config openssl.cnf
openssl x509 -req -in ca/ca.csr -out ca/ca.crt -signkey ca/ca.key -days 365

2. generate server certificate

here, skipped the steps to generate server.csr and server.key file
openssl x509 -req -in server/server.csr -out server/server.crt -signkey server/server.key -CA ca/ca.crt -CAkey ca/ca.key -CAcreateserial -days 365

3. generate client certificate

here, skipped the steps to generate client.csr and client.key file
openssl x509 -req -in client/client.csr -out client/client.crt -signkey client/client.key -CA ca/ca.crt -CAkey ca/ca.key -CAcreateserial -days 365

Then I configured filebeat tls section like below:

logstash:
hosts: ["newname3:5044"]
tls:
certificate_authorities: ["/usr/ssl/ca/ca.crt"]
certificate: "/usr/ssl/client/client.crt"
certificate_key: "/usr/ssl/client/client.key"

logstash configuration:

input {
beats {
ssl_certificate_authorities => ["/usr/ssl/ca/ca.crt"]
ssl_certificate => "/usr/ssl/server/server.crt"
ssl_key => "/usr/ssl/server/server.key"
ssl_verify_mode => "force_peer"
port => 5044
ssl => true
}
}

By the way, my filebeat and logstash are installed in the same virtual machine. filebeat version is 1.2.3, and logstash is 2.3.4
I searched a lot, but didn't find an applicable solution for me. Can anyone who met this kind of issue before, or who have successfully setup filebeat TLS, give me some tips? Thank you so much.

[caoping]

After I did some investigation, I can run "curl -v --cert ... https://localhost:5044" to validate my certificate. But when starting filebeat, different error occurred.

2016/08/03 09:46:28.901034 transport.go:125: ERR SSL client failed to connect with: x509: certificate signed by unknown authority (possibly because of "x509: cannot verify signature: algorithm unimplemented" while trying to verify candidate authority certificate "My CA")

I think I made a small progress although I can't configure it successfully. Can anyone give me some tips on that issue?

By the way, below is a part of response message, hope it's helpful for trouble shooting.

SSL connection using TLS_DHE_RSA_WITH_AES_128_CBC_SHA

[ruflin]

Did you follow the guide here? https://www.elastic.co/guide/en/beats/filebeat/current/configuring-tls-logstash.html

[steffens (Steffen Siering)]

the error message is very interesting no? Error message says it can not read the certificate, because the algorithm used to sign the certificate is not supported by golang. What's the signature algorithm used for the certificate?

Something like
$ openssl x509 -in <certificate-file> -noout -text
should print Signature Algorithm: ... .

Here is a list of supposedly supported algorithms: https://golang.org/pkg/crypto/x509/#SignatureAlgorithm

[caoping]

@steffens
Thanks for your reply. Below is the algorithm used.

The command to generate key file is:

openssl genrsa -des3 -out key.pem 1024

I checked the url you post, there is only md5withrsa, not what I used, md5withrsaencryption. That's probably the cause, right? If that, what I should use? Remove "-des3" option?

[caoping]

Hi @ruflin,

Thanks for your reply.

Yes, I followed the guide that you mentioned. The certificate_authority in filebeat and logstash configuration, is the self-signed root CA. Also, I used that root ca to sign the filebeat certificate and logstash certificate.

I'm trying to not encrypt the generated key. Hope it works. Any update, I'll post here. Thank you again.

[caoping]

Hi @steffens, I tried to generate key file without "-des3" option, then use this key file to generate my root ca. However, the algorithm printed is still the same, "md5withrsaencryption". How can I change the algorithm? And my direction is correct?

[caoping]

Hi @steffens

Can I use keytool to generate certificate?

I just had a try, and found the signature algorithm used without the "Encryption" suffix. I didn't find how to change the signature algorithm when using openssl, and it always use the algorithms with "Encryption" suffix, which are not supported by Golang, per the link you posted.

[caoping]

Hi @ruflin

Still not work. I followed the guide you mentioned to configure filebeat and logstash. But when validating certificate, the curl command listed there doesn't work for me. Finally, I used below command, and it seems the validation succeeded.

curl -v --cert /mycert.pem --key /mycertkey.pem https://localhost:5044

This is results of above command:

However, it still failed when starting filebeat after above validation. The error message is "signature algorithm unimplemented". By looking into the certificate, it said the algorithm is "MD5WithRsaEncryption".

I'd like to know if that's a supported algorithm. If not, how can I change it to a supported one? I didn't find how to change it when using openssl.

This TLS issue really blocked me for several days. Any further suggestion is really appreciated. Thanks a lot!

[ruflin]

As far as I know we support the algorithm supported by Golang. The list can be found here: https://golang.org/pkg/crypto/x509/#SignatureAlgorithm There seems to be a MD5WithRSA in the list. Perhaps @andrewkroh has some more ideas here?

[andrewkroh]

This is probably unrelated to the problem, but the first thing I noticed is that you are using MD5. Use SHA256 instead. Most people stopped using MD5 years ago. https://www.kb.cert.org/vuls/id/836068

[steffens (Steffen Siering)]

I'm a little confused about this line:

$ curl -v --cert /mycert.pem --key /mycertkey.pem https://localhost:5044

Our integration tests use a self-signed certificate only. Can you test this approach first, before adding a CA to the picture?
See gencerts.sh from you logstash integration tests, on how a self-signed certificate for testing is created. The testing hostname is logstash. Change -subj '/CN=logstash/' to your hostname in use. See logstash input config using the generated certificate.

If this works, let's continue with server certificate + CA only (no client authentication).

[caoping]

Hi @steffens

I generated a certificate for logstash server referring to gencerts.sh, and also added it in the logstash config, like below.
input { beats { ssl_certificate => "/etc/pki/tls/certs/logstash.crt" ssl_key => "/etc/pki/tls/private/logstash.key" port => 5044 ssl => true } }
But I don't know how to configure filebeat. Should I enable tls in logstash output?

[caoping]

Hi @steffens,

I enabled tls section, like this. Same .crt and .key files with logstash configuration as filebeat and logstash are installed in the same VM.

 tls:
      certificate_authorities: ["/etc/pki/tls/certs/logstash.crt"]
      certificate: "/etc/pki/tls/certs/logstash.crt"
      certificate_key: "/etc/pki/tls/private/logstash.key"

It's a good news that no error log when starting filebeat. But also no log showing it's using encrypted way to communicate with logstash. Does it mean it's working?

[steffens (Steffen Siering)]

As config according to gencerts.sh is using a self-signed certificate only with client only validating server this config is enough:

 tls:
      certificate_authorities: ["/etc/pki/tls/certs/logstash.crt"]

The key file is required to parse the private part of a certificate.

Right, there is no log message if encryption is used. But I think it's a great idea to print some connection information if TLS is used. I created this ticket to implement some logs: INFO log if TLS is used · Issue #2198 · elastic/beats · GitHub

If all you want is encryption, you can stop here, but this is far from the initial solution. You have had a CA + client authentication. In case you just need client authentication you can create a self-signed client certificate just as you did for logstash. Given we name the files filebeat.crt and filebeat.key your config will look like:

logstash.conf:

input {
   beats {
      port => 5044
      ssl => true
      ssl_certificate => "/etc/pki/tls/certs/logstash.crt"
      ssl_key => "/etc/pki/tls/private/logstash.key"
      ssl_certificate_authorities => ["/etc/pki/tls/certs/filebeat.crt"]
      ssl_verify_mode => "force_peer"
    }
}

and filebeat.yml:

    tls:
      certificate:  "/etc/pki/tls/certs/filebeat.crt"
      certificate_key: "/etc/pki/client/filebeat.key"
      certificate_authorities: ["/etc/pki/tls/certs/logstash.crt"]

As you can see, it becomes very cumbersome to add many more certificates for additional filebeat instances. This is where a CA comes into play.

Next let's add a CA (certificate authority) and a server certificate only. For example see this guide (I haven't tested this myself, though). As explained in the guide, it's good practice to create intermediate signing CAs. Unfortunately there's a bug in SSL libs loading intermediate CAs. From docs:

Although intermediate CAs are currently not supported, you may be able to work around this issue by merging all the certificates in the chain into one file. You can create the PEM file containing the CA chain by concatenating the root CA cert and the intermediate CA into a single file: cat root-ca.crt intermediate-ca.crt > chain.crt. Then set certificate_authorities to use this file: certificate_authorities: ['chain.crt'].

See beats->logstash TLS docs

[caoping]

All I want is encrypted communication between filebeat and logstash. So the configuration you provided is enough, right? If yes, that's great. Thank you so much.

[steffens (Steffen Siering)]

Yes, this config is enough for encrypted traffic, only. Without client authentication, any client can connect and push events though.