I have spun up an Elastic Cloud on Kubernetes cluster with Let's Encrypt SSL certificates with a domain name I own. All of these SSL certs are running properly, as well as my cluster. I have tested this locally and publicly.
When trying to install Elastic Agent on a Ubuntu 20.0.4 agent on my local network (happens to be the same subnet (10.10.2.1/24) as the ECK cluster) I am unable to get proper communication between the agent and the elastic cluster. When installing, I run:
corey@k3s-node-1:/elastic-agent-7.11.0-linux-x86_64$ sudo ./elastic-agent install -f --kibana-url=https://elastic.domain.example:5602 --enrollment-token=<redacted>
After running the command, and getting a positive return message. The agent shows up in https://elastic.domain.example/app/fleet#/fleet/agents for a single "heartbeat" but does not ship any data (yes there is an assigned integration, linux). Upon looking into the log files, I noticed that it was trying to reach out to https://0.0.0.0:5601 (which since it is one of the nodes running my kubernetes stack is technically a valid way to reach kibana) however it errors out with a certificate error (expected) since 0.0.0.0 is not a valid domain under the valid cert. I dug into the fleet.yml file to look into the configuration of the agent and noticed that 0.0.0.0:5601 was listed under hosts:. I do not understand how this got there with the fresh installation listed, or why this agent is reaching out over that when I specified the Kibana url for it to use. Please let me if there is a way I can remove this, and force it to use the proper URL or if there is somehow a reason it is trying to do this. I have debugged this for a while with no luck. Thank you!
The errors in elastic-agent.log.1 are:
2021-02-12T21:08:31.363Z ERROR application/fleet_gateway.go:168 failed to dispatch actions, error: acknowledge 1 actions '[action_id: 35cb0910-6d72-11eb-83f4-45fadbc24bda, type: POLICY_CHANGE]' for elastic-agent '53d5a150-6d76-11eb-83f4-45fadbc24bda' failed: fail to ack to fleet: Post "https://0.0.0.0:5601/api/fleet/agents/53d5a150-6d76-11eb-83f4-45fadbc24bda/acks?": x509: cannot validate certificate for 0.0.0.0 because it doesn't contain any IP SANs
2021-02-12T21:29:40.458Z ERROR application/fleet_gateway.go:187 Could not communicate with Checking API will retry, error: fail to checkin to fleet: Post "https://0.0.0.0:5601/api/fleet/agents/53d5a150-6d76-11eb-83f4-45fadbc24bda/checkin?": x509: cannot validate certificate for 0.0.0.0 because it doesn't contain any IP SANs
The fleet.yml contains:
corey@k3s-node-1:$ sudo cat /opt/Elastic/Agent/fleet.yml
agent:
id: <not sure if this needs to be redacted, but it is>
fleet:
enabled: true
access_api_key: <redacted>
kibana:
protocol: https
host: elastic.domain.example:5601
hosts:
- 0.0.0.0:5601
timeout: 5m0s
reporting:
threshold: 10000
check_frequency_sec: 30
agent:
id: ""