Root cause was due to the Output parameters not being fully set/defined for my Elasticsearch host. These can be manually configured in the "Fleet...Settings...Output...Edit output" section in Kibana which is ultimately what I did to solve my issue.
I couldn't find any environment variables to specify to docker which would provide either the Elasticsearch host url or the CA to use when connecting to the Elasticsearch node which would then pre-populate this detail in the Kibana Fleet Settings.
I did locate this post which clarified that an undocumented environment variable called FLEET_SERVER_ELASTICSEARCH_CA_TRUSTED_FINGERPRINT
exists which I believe is the same as the Elasticsearch CA trusted fingerprint
parameter showed in the same "Kibana...Settings...Output...Edit output" screen (as below).
My new docker config...
> docker run -d --name elasticagent --hostname elasticagent --restart always --user root \
> --network blacknet10 --ip xxx.xxx.xxx.xxx \
> -v /var/run/docker.sock:/var/run/docker.sock \
> -v /volume1/docker/traefik/logs:/tmp/traefik \
> -v /volume1/docker/elasticsearch/certs:/tmp/certs/ca \
> -e "FLEET_ENROLL=1" \
> -e "FLEET_URL=https://mydomain:8220" \
> -e "FLEET_SERVER_ENABLE=true" \
> -e "FLEET_SERVER_ELASTICSEARCH_HOST=https://mydomain:9200" \
> -e "FLEET_SERVER_SERVICE_TOKEN=mytoaken" \
> -e "FLEET_SERVER_ELASTICSEARCH_CA=/tmp/certs/ca/elasticsearch-ca.pem" \
> -e "FLEET_SERVER_POLICY=fleet-server-policy" \
> docker.elastic.co/beats/elastic-agent:8.2.0
I couldn't get the FLEET_SERVER_ELASTICSEARCH_CA_TRUSTED_FINGERPRINT
environment variable to work, I believe I ran into a bug being discussed here.
Lastly it would seem the environement variable FLEET_SERVER_ELASTICSEARCH_CA
is necessary for fleet server specifications only and isn't used by the metricbeat component of the agent. As such when you inspect the Elastic-Agent logs at /usr/share/elastic-agent/state/data/logs/default
you can see the errors about the x509 cert being signed by an unknown authority. To fix this I had to add the yaml config shown in the above screen snip to pass the ssl.certificate_authorities
parameter. You can see the edits you make to the "Fleet...Settings...Output...Edit output" by inspecting the /usr/share/elastic-agent/state/data/state.yml
file inside the container.