I've been running 7.12 with 10 test endpoints all operating in Fleet without any problem.
After upgrading to 7.13, I'm trying to re-deploy to the test endpoints (after manually removing the previous agent on each endpoint) however, the Fleet server won't start and produces the following output:
sudo ./elastic-agent install -f --fleet-server-es=https://elastic.test.local:9200 --fleet-server-service-token=AAE-deleted-QQ
The Elastic Agent is currently in BETA and should not be used in production
2021-06-01T10:18:01.989Z INFO cmd/enroll_cmd.go:300 Generating self-signed certificate for Fleet Server
2021-06-01T10:18:03.527Z INFO cmd/enroll_cmd.go:610 Waiting for Elastic Agent to start Fleet Server
2021-06-01T10:18:07.535Z INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-06-01T10:18:08.536Z INFO cmd/enroll_cmd.go:643 Fleet Server - Error - x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "Elastic Certificate Tool Autogenerated CA")
2021-06-01T10:18:14.547Z INFO cmd/enroll_cmd.go:648 Fleet Server - Error - x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "Elastic Certificate Tool Autogenerated CA")
...
...
2021-06-01T10:19:13.654Z INFO cmd/enroll_cmd.go:643 Fleet Server - Restarting
2021-06-01T10:19:14.656Z INFO cmd/enroll_cmd.go:643 Fleet Server - Error - x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "Elastic Certificate Tool Autogenerated CA")
2021-06-01T10:19:20.665Z INFO cmd/enroll_cmd.go:648 Fleet Server - Error - x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "Elastic Certificate Tool Autogenerated CA")
2021-06-01T10:19:24.674Z INFO cmd/enroll_cmd.go:643 Fleet Server - Restarting
2021-06-01T10:19:25.675Z INFO cmd/enroll_cmd.go:643 Fleet Server - Error - x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "Elastic Certificate Tool Autogenerated CA")
2021-06-01T10:19:31.684Z INFO cmd/enroll_cmd.go:648 Fleet Server - Error - x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "Elastic Certificate Tool Autogenerated CA")
...
...
2021-06-01T10:19:57.722Z INFO cmd/enroll_cmd.go:643 Fleet Server - Restarting
2021-06-01T10:19:58.723Z INFO cmd/enroll_cmd.go:643 Fleet Server - Error - x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "Elastic Certificate Tool Autogenerated CA")
Error: fleet-server never started by elastic-agent daemon: context canceled
Error: enroll command failed with exit code: 1
I have an locally autogenerated ca.crt, which is registered in the local certifcate store - all other TLS/xpack certificate security aspects of this host work fine.
I get these errors when trying to use the local Elasticsearch host as the Fleet server (my preferred option, as it's always up) and the same error on a Windows host - which also has the root ca certificate in the trusted root store, and can access Kibana without generating errors.
I'm not sure where to go next - certificates all worked perfectly fine until the upgrade.
Cheers,
John.