Hello guys,
I'm here because i got some troubles while enabling TLS between filebeat and logstash
For a little context I'm all my machines run under debian 11, i got 3 servers, first is filebeat who communicate with a logstash server using TLS who communicate with an Elasticsearch and kibana server using tls.
For filebeat and logstash i generated openssl ca, certificate and key,
when im trying to enable certificate on my filebeat server i got this response:
root@debiandefault:/etc/filebeat# curl -v --cacert /etc/filebeat/ca.pem https://***.***.***.***:5044
* Trying ***.***.***.***:5044...
* Connected to ***.***.***.*** (***.***.***.***) port 5044 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/filebeat/ca.pem
* CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Request CERT (13):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: CN=Logstashcert
* start date: Dec 6 15:31:38 2022 GMT
* expire date: Dec 6 15:31:38 2023 GMT
* subjectAltName: host "***.***.***.***" matched cert's IP address!
* issuer: C=AU; ST=Some-State; O=Internet Widgits Pty Ltd
* SSL certificate verify ok.
> GET / HTTP/1.1
> Host: ***.***.***.***:5044
> User-Agent: curl/7.74.0
> Accept: */*
>
* TLSv1.3 (IN), TLS alert, bad certificate (554):
* OpenSSL SSL_read: error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate, errno 0
* Closing connection 0
curl: (56) OpenSSL SSL_read: error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate, errno 0
there is my filebeat.yml:
# ------------------------------ Logstash Output -------------------------------
output.logstash:
# The Logstash hosts
hosts: [***.***.***.***:5044"]
# ssl:
# certificate_authorities: ["/etc/filebeat/logstash.crt"]
# Optional SSL. By default is off.
# List of root certificates for HTTPS server verifications
ssl.certificate_authorities: ["/etc/filebeat/ca.pem"]
# Certificate for SSL client authentication
ssl.certificate: "/etc/filebeat/cert.pem"
# Client Certificate Key
ssl.key: "/etc/filebeat/cert-key-pkcs8.key"
and my logstash conf.d even if i think there is no problem here, everything is fine in the plain log
GNU nano 5.4 /etc/logstash/conf.d/nginx.conf
input {
beats {
port => 5044
ssl => true
ssl_certificate_authorities => ["/etc/logstash/ca.pem"]
ssl_certificate => "/etc/logstash/cert.pem"
ssl_key => "/etc/logstash/cert-key-pkcs8.key"
ssl_verify_mode => "force_peer"
}
}
filter {
grok {
patterns_dir => ["/etc/logstash/pattern"]
match => { "message" => "%{IPORHOST:clientip} %{NGUSER:ident} %{NGUSER:auth} \[%{HTTPDATE:timestamp}\] \"%{WORD:verb} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}\" %{NUMBER:response}" }
}
}
output {
elasticsearch {
hosts => ["https://***.***.***.***:9200"]
index => "nginx-%{+YYYY.MM.dd}"
user => "**********"
password => "************"
ssl => true
ssl_certificate_verification => true
cacert => "/usr/local/etc/ssl/certs/elastic/http_ca.crt"
data_stream => "false"
}
}
Of course i can't get log in kibana and im sure that the the problem is between filebeat and logstash because when in logstash i set local input i get logs in kibana
I do not really know what to do i didn't know well ssl certificates... someone know where the issue is from please ?
Sorry for my bad english
Thanks for reading