I have managed to implement an SSL connection between a FileBeat (8.13) agent and a LogStash (8.13) but have noticed in the FileBeat syslog logs that the SSL connections to the LogStash server are getting reset and these entries are repeated after a few entries are shipped to the LogStash server;
<30>1 2024-06-12T08:55:00.082579-07:00 svsandd2 systemd 1 - - Starting system activity accounting tool...
<30>1 2024-06-12T08:55:00.110172-07:00 svsandd2 systemd 1 - - sysstat-collect.service: Succeeded.
<30>1 2024-06-12T08:55:00.110305-07:00 svsandd2 systemd 1 - - Started system activity accounting tool.
<131>1 2024-06-12T08:55:12.617865-07:00 svsandd2 filebeat 13788 - - 2024-06-12T08:55:12.617-0700 ERROR [logstash] map[file.line:100 file.name:logstash/sync.go function:github.com/elastic/beats/v7/libbeat/outputs/logstash.(*syncClient).reconnect] error closing connection to logstash host <redacted>:7514: tls: failed to send closeNotify alert (but connection was closed anyway): write tcp <redacted>:54892->*<redacted>*:7514: write: connection reset by peer, reconnecting...#011{"ecs.version": "1.6.0"}
<131>1 2024-06-12T08:55:23.859990-07:00 svsandd2 filebeat 13788 - - 2024-06-12T08:55:23.859-0700 ERROR [logstash] map[file.line:100 file.name:logstash/sync.go function:github.com/elastic/beats/v7/libbeat/outputs/logstash.(*syncClient).reconnect] error closing connection to logstash host <redacted>:7514: tls: failed to send closeNotify alert (but connection was closed anyway): write tcp <redacted>:42916->*<redacted>*:7514: write: connection reset by peer, reconnecting...#011{"ecs.version": "1.6.0"}
Nevertheless, the logs are being shipped as expected, so the actual SSL handling is working.
Herewith the relevant configuration snippets;
/etc/filebeat/filebeat.yml
output.logstash:
hosts:
- "*<redacted>*:7514"
- "*<redacted>*:7514"
loadbalance: true
ttl: 30
pipelining: 0
ssl:
enabled: true
key: /etc/pki/tls/private/filebeat.key
certificate: /etc/pki/tls/certs/filebeat.pem
certificate_authorities: [ /etc/pki/ca-trust/source/anchors/*<redacted>*.CA_crt ]
verification_mode: strict
/etc/logstash/conf.d/pipeline.conf
input {
# listen for FileBeat log objects on the internal network interface
beats {
host => "127.0.0.1"
port => 7512
}
beats {
port => 7514
ssl_enabled => true
ssl_key => "/etc/pki/tls/private/logstash.key"
ssl_certificate => "/etc/pki/tls/certs/logstash.pem"
ssl_certificate_authorities => [ "/etc/pki/ca-trust/source/anchors/*<redacted>*.CA_crt" ]
ssl_client_authentication => required
}
}
output {
# dump entry to /var/log/logstash/pipeline.output
file {
path => "/var/log/logstash/pipeline.output"
}
# discard output to /dev/null
# null {}
}
I am uncertain what is happening here and whether it is a configuration or an implementation issue but it is disconcerting as it is happening frequently. Any suggestions on how fix the issue to stop these disconcerting messages would be much appreciated.