Kibana version: 7.17.4
Elasticsearch version: 7.17.4
APM Server version: 7.17.4
APM Agent language and version: Java APM agent 1.30.0
Browser version: Chrome
Original install method (e.g. download page, yum, deb, from source, etc.) and version: k8s deployment in azure
Fresh install or upgraded from other version? Fresh
Is there anything special in your setup? For example, are you using the Logstash or Kafka outputs? Are you using a load balancer in front of the APM Servers? Have you changed index pattern, generated custom templates, changed agent configuration etc. NO
I have tried Elastic APM with ELK stack 8.2.0 and java apm agent 1.30.0. It worked well without auth issue. I have disabled SSL and was able to communicate successfully with Elasticsearch from APM Server by just providing username and password. ("elastic"/"elastic").
For some reason, I need to downgrade with version 7.17.4 with the same java apm agent 1.30.0. Here I am getting auth issue, its expecting auth headers, but the same works with 8.2.0
Why i'm not able to communicate with SUPER USER (elastic/elastic).
If the auth headers are MUST, then please suggest me a doc for procedures.
Please find below apm server yaml file and logs for reference
apm server yaml file
apiVersion: apm.k8s.elastic.co/v1
kind: ApmServer
metadata:
name: {{ .Release.Name }}
namespace: {{ .Release.Namespace }}
spec:
type: ApmServer
version: {{ .Values.elasticStackVersion }}
count: 1
elasticsearchRef:
name: {{ .Release.Name }}
kibanaRef:
name: {{ .Release.Name }}
config:
output:
elasticsearch:
username: "elastic"
password: "elastic"
logging.level: info
logging.to_files: true
logging.files:
path: /var/log/apm-server
name: apm-server
keepfiles: 7
permissions: 0644
podTemplate:
spec:
containers:
- name: apm-server
resources:
limits:
memory: 4Gi
cpu: 1
apm-agent log
2022-06-16 08:35:40,230 [main] INFO co.elastic.apm.agent.configuration.StartupInfo - Starting Elastic APM 1.30.0 as apm-translation-service (0.0.3-SNAPSHOT) on Java 17.0.2 Runtime version: 17.0.2+8-86 VM version: 17.0.2+8-86 (Oracle Corporation) Linux 5.4.0-1069-azure
2022-06-16 08:35:40,231 [main] INFO co.elastic.apm.agent.configuration.StartupInfo - service_name: 'apm-translation-service' (source: Java System Properties)
2022-06-16 08:35:40,232 [main] INFO co.elastic.apm.agent.configuration.StartupInfo - server_url: 'https://ip:8200' (source: Java System Properties)
2022-06-16 08:35:40,232 [main] INFO co.elastic.apm.agent.configuration.StartupInfo - verify_server_cert: 'false' (source: Java System Properties)
2022-06-16 08:35:40,232 [main] INFO co.elastic.apm.agent.configuration.StartupInfo - application_packages: 'com.idit.translation' (source: Java System Properties)
2022-06-16 08:35:47,414 [main] INFO co.elastic.apm.agent.impl.ElasticApmTracer - Tracer switched to RUNNING state
2022-06-16 08:35:49,930 [elastic-apm-server-healthcheck] INFO co.elastic.apm.agent.report.ApmServerHealthChecker - Elastic APM server is available:
2022-06-16 08:35:50,010 [elastic-apm-server-healthcheck] WARN co.elastic.apm.agent.report.ApmServerHealthChecker - Failed to parse version of APM server https://ip:8200/: Unexpected end of JSON input
2022-06-16 08:35:50,017 [elastic-apm-remote-config-poller] ERROR co.elastic.apm.agent.configuration.ApmServerConfigurationSource - Unexpected status 401 while fetching configuration
apm server log
{"log.level":"error","@timestamp":"2022-06-17T04:23:01.558Z","log.logger":"request","log.origin":{"file.name":"middleware/log_middleware.go","file.line":60},"message":"authentication failed: missing or improperly formatted Authorization header: expected 'Authorization: Bearer secret_token' or 'Authorization: ApiKey base64(API key ID:API key)'","service.name":"apm-server","url.original":"/intake/v2/events","http.request.method":"POST","user_agent.original":"apm-agent-java/1.30.0 (apm-translation-service 0.0.3-SNAPSHOT)","source.address":"ip","http.request.id":"6bb08034-529e-4457-a78c-e76db4a129ed","event.duration":116701,"http.response.status_code":401,"error.message":"authentication failed: missing or improperly formatted Authorization header: expected 'Authorization: Bearer secret_token' or 'Authorization: ApiKey base64(API key ID:API key)'","ecs.version":"1.6.0"}