Failed to verify server certificate - APM Go Agent with ECK

Elastic Observability APM
,
7

Hi @stephenb thank you so much for the help!

Sorry I missed this sentence early:

The agent is going to need the apm-token as well that will be in hm-apm-apm-token

Here are 3 ways I tested that work:

First to get ELASTIC_APM_SECRET_TOKEN, run

kubectl get secret hm-apm-apm-token --namespace=elastic --output=go-template='{{index .data "secret-token"}}' | base64 -d && echo

Method 1

Change to ELASTIC_APM_VERIFY_SERVER_CERT: "false" and remove ELASTIC_APM_SERVER_CERT, so like

ELASTIC_APM_SERVER_URL: "https://hm-apm-apm-http.elastic:8200"
ELASTIC_APM_ENVIRONMENT: "development"
ELASTIC_APM_LOG_LEVEL: "debug"
ELASTIC_APM_LOG_FILE: "stderr"
ELASTIC_APM_SECRET_TOKEN: "xxx"
ELASTIC_APM_VERIFY_SERVER_CERT: "false"

Method 2

kubectl get secret hm-apm-apm-http-certs-public --namespace=elastic --output=go-template='{{index .data "tls.crt" | base64decode }}' > data/elastic-apm/tls.crt

Mounted tls.crt to the app, and pass these to Go app envs

ELASTIC_APM_SERVER_URL: "https://hm-apm-apm-http.elastic:8200"
ELASTIC_APM_ENVIRONMENT: "development"
ELASTIC_APM_LOG_LEVEL: "debug"
ELASTIC_APM_LOG_FILE: "stderr"
ELASTIC_APM_SECRET_TOKEN: "xxx"
ELASTIC_APM_SERVER_CERT: "/data/elastic-apm/tls.crt"

Method 3

Adding ELASTIC_APM_SERVER_CA_CERT_FILE will still work:

kubectl get secret hm-apm-apm-http-certs-public --namespace=elastic --output=go-template='{{index .data "tls.crt" | base64decode }}' > data/elastic-apm/tls.crt
kubectl get secret hm-apm-apm-http-certs-public --namespace=elastic --output=go-template='{{index .data "ca.crt" | base64decode }}' > data/elastic-apm/ca.crt

Mounted both tls.crt and ca.crt to the app, and pass these to Go app envs

ELASTIC_APM_SERVER_URL: "https://hm-apm-apm-http.elastic:8200"
ELASTIC_APM_ENVIRONMENT: "development"
ELASTIC_APM_LOG_LEVEL: "debug"
ELASTIC_APM_LOG_FILE: "stderr"
ELASTIC_APM_SECRET_TOKEN: "xxx"
ELASTIC_APM_SERVER_CERT: "/data/elastic-apm/tls.crt"
ELASTIC_APM_SERVER_CA_CERT_FILE: "/data/elastic-apm/ca.crt"
1 Like