[apm-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Error sending data to APM server: cannot retry due to server authentication, in streaming mode, response code is 401

[kumar_kung]

Kibana version : 7.5.2

Elasticsearch version : 7.5.2

APM Server version : 7.5.2

**APM Agent language **: java

• Original install method : ECK operator 1.0.0 GA

I have updated apm-server-quickstart-apm-token secret-token to a known value, but the apm agent is unable to connect to apm server, throwing authentication error. Please let me know which secret should i update for apm server to accept secret-token i have set

[Eyal_Koren]

Take a look at this. The specified command should get you the secret token.
Make sure it is Base-64-encoded and stored in the right field. Then make sure the agent actually uses the right token (or retrieves it from the right secret, if using the approach suggested in the blog post).

[kumar_kung]

Thanks @Eyal_Koren

kubectl get secret/apm-server-quickstart-apm-token -o go-template='{{index .data "secret-token" | base64decode}}' -n elastic-system
output: elasticuser

-Delastic.apm.secret_token=elasticuser

apm-server-quickstart-apm-token and elastic.apm.secret_token are elasticuser, yet i am getting authentication issue.

[Eyal_Koren]

Did you configure the APM server to use the same token as well?

[kumar_kung]

@Eyal_Koren

using

kubectl get secret/apm-server-quickstart-apm-token -o go-template='{{index .data "secret-token" | base64decode}}' -n elastic-system
output: elasticuser

-Delastic.apm.secret_token=elasticuser

any value set in apm-server-quickstart-apm-token secret will be used by apm-sever right? doesnt it automatically take it, if not pls let me know where to make change for secret token to be used by apm-server in eck

find apm-server-quickstart-apm-config:
apm-server:
host: :8200
secret_token: ${SECRET_TOKEN}

[axw (Andrew Wilkins)]

Did you set that secret value after creating the ApmServer resource? If so, I suspect you're hitting https://github.com/elastic/cloud-on-k8s/issues/523. Until that's addressed, if you delete the apm-server pod then the new one should pick up the updated secret token.

[kumar_kung]

Yes, i was creating secret post apm server resource creation , thanks