Hi everyone,
I've been wanting to try elastic-agent for quite some time now so I booted up a machine and tried to configure everything, but I've encountered an error that for the love of me I simply cannot fix.
I have followed this guide to setup ELK in docker with TLS enabled.
Once that was done, I could enter into Kibana without problem.
I proceeded at checking the documentation for starting a Fleet server as described here under the "self-managed" tab.
I downloaded the elastic-agent, selected the "quick start" option and run the install command, which looks something like this (from the doc):
sudo ./elastic-agent install -f \
--fleet-server-es=https://localhost:9200 \
--fleet-server-service-token=AAEbAWVsYXN0aWMvZmxlaXQtc2VydmVzL3Rva2VuLTE2MeIzNTY1NTQ3Mji6dERXeE9XbW5RRTZqNlJMWEdIRzAtZw \
--fleet-server-policy=27467ed1-1bfd-11ec-9b88-a7c3d83e2897
Here I encountered the first issue: the elastic-agent doesn't like the certificate that was created by following the documentation on setting up ELK in docker with TLS.
I thus looked around and changed the above command with:
sudo ./elastic-agent install -f \
--fleet-server-es=https://localhost:9200 \
--fleet-server-service-token=AAEbAWVsYXN0aWMvZmxlaXQtc2VydmVzL3Rva2VuLTE2MeIzNTY1NTQ3Mji6dERXeE9XbW5RRTZqNlJMWEdIRzAtZw \
--fleet-server-policy=27467ed1-1bfd-11ec-9b88-a7c3d83e2897
--fleet-server-es-ca=/var/snap/docker/common/var-lib-docker/volumes/es_certs/_data/ca/ca.crt
so that the elastic-agent could validate the server certificate provided by Elasticsearch.
(Shouldn't the documentation be updated?)
With the above command, I can see the agent popping up in Kibana however, no logs is sent to Elasticsearch and no data stream is there.
Checking the logs of the Elasticsearch es01 container, I see a tons of
{"type": "server", "timestamp": "2021-10-16T14:36:15,343Z", "level": "WARN", "component": "o.e.h.AbstractHttpServerTransport", "cluster.name": "es-docker-cluster", "node.name": "es01", "message": "caught exception while handling client http traffic, closing connection Netty4HttpChannel{localAddress=/172.25.0.3:9200, remoteAddress=/172.25.0.1:34074}", "cluster.uuid": "T_2o0sO_SfSuwMUvKdTRmg", "node.id": "imQywT40R66wUI_khsT-Gg" ,
"stacktrace": ["io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate"
If I get this right, it looks like that Elasticsearch cannot validate the certificate provided by the elastic-agent? Meaning that there's a mutual authentication where Elasticsearch tries to authenticate the elastic-agent?
