Ssl_certificate_authorities seems to have no effect

[hexoffender (Robert Paul)]

I'm getting an error in filebeat:

ERROR [publisher_pipeline_output] pipeline/output.go:154 Failed to connect to backoff(async(tcp://localhost:5044)): x509: certificate signed by unknown authority

Here is my logstash config:

input {
  beats {
    port => 5044
    host => "localhost"
    ssl => true
    ssl_certificate_authorities => ["/etc/ssl/ca.crt"]
    ssl_certificate => "/etc/ssl/server.crt"
    ssl_key => "/etc/ssl/server.pk8"
    ssl_verify_mode => "force_peer"
  }
}

And here is my filebeat config:

- type: log

  enabled: true

  paths:
    - /var/log/nginx/access.log

output.logstash:
  hosts: ["localhost:5044"]
  ssl.verification_mode: "certificate"
  ssl.certificate_authorities: ["/etc/ssl/ca.crt"]
  ssl.certificate: "/etc/ssl/client.crt"
  ssl.key: "/etc/ssl/client.key"

It seems the ssl.certificate_authorities options in both configs have no effect. Here is how I generated the certificates:

openssl genrsa -aes256 -out ca.key 4096
openssl req -key ca.key -new -x509 -days 7300 -sha256 -extensions v3_ca -out ca.crt
openssl genrsa -out server.key 2048
openssl req -new -key server.key -out server.csr
openssl x509 -req -days 3650 -in server.csr -signkey server.key -out server.crt -CA ca.crt -CAkey ca.key -set_serial 1
openssl genrsa -out client.key 2048
openssl req -new -key client.key -out client.csr
openssl x509 -req -days 3650 -in client.csr -signkey client.key -out client.crt -CA ca.crt -CAkey ca.key -set_serial 1
openssl pkcs8 -topk8 -in server.key -nocrypt -inform PEM -outform PEM -out server.pk8

So the certificates were generated both from the same CA key, and I'm pointing the config to that key but I'm still getting x509: certificate signed by unknown authority. Am I missing a step here for getting this working with self signed certificates?

The documentation here for verification_mode specifies that the certificate verification mode Verifies that the provided certificate is signed by a trusted authority (CA), but does not perform any hostname verification. So this should be a working configuration. Did I miss a step here?

[hexoffender (Robert Paul)]

I figured it out. The way I was generating certificates wasn't enough. You need to make sure the keys have the correct Key Usage and Extended Key Usage.

The correct way to generate the certificates is as follows:
server.conf:

[req]
distinguished_name = req_distinguished_name
req_extensions = v3_req
prompt = no

[req_distinguished_name]
countryName                     = XX
stateOrProvinceName             = XXXXXX
localityName                    = XXXXXX
postalCode                      = XXXXXX
organizationName                = XXXXXX
organizationalUnitName          = XXXXXX
commonName                      = XXXXXX
emailAddress                    = XXXXXX

[v3_req]
keyUsage = keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names

[alt_names]
DNS.1 = DOMAIN_1
DNS.2 = DOMAIN_2
DNS.3 = DOMAIN_3
DNS.4 = DOMAIN_4

openssl genrsa -out server.key 2048
openssl req -sha512 -new -key server.key -out server.csr -config server.conf
echo "C2E9862A0DA8E970" > serial
openssl x509 -days 3650 -req -sha512 -in server.csr -CAserial serial -CA ca.crt -CAkey ca.key -out server.crt -extensions v3_req -extfile server.conf
mv server.key server.key.pem && openssl pkcs8 -in server.key.pem -topk8 -nocrypt -out server.key

client.conf

[req]
distinguished_name = req_distinguished_name
req_extensions = v3_req
prompt = no
 
[req_distinguished_name]
countryName                     = XX
stateOrProvinceName             = XXXXXX
localityName                    = XXXXXX
postalCode                      = XXXXXX
organizationName                = XXXXXX
organizationalUnitName          = XXXXXX
commonName                      = XXXXXX
emailAddress                    = XXXXXX

[ usr_cert ]
# Extensions for server certificates (`man x509v3_config`).
basicConstraints = CA:FALSE
nsCertType = client, server
nsComment = "OpenSSL FileBeat Server / Client Certificate"
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer:always
keyUsage = critical, digitalSignature, keyEncipherment, keyAgreement, nonRepudiation
extendedKeyUsage = serverAuth, clientAuth

[v3_req]
keyUsage = keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth, clientAuth

openssl genrsa -out client.key 2048
openssl req -sha512 -new -key client.key -out client.csr -config client.conf
openssl x509 -days 3650 -req -sha512 -in client.csr -CAserial serial -CA ca.crt -CAkey ca.key -out client.crt -extensions v3_req -extensions usr_cert  -extfile client.conf