Kibana version: 8.0.0
Elasticsearch version: 8.0.0
APM Server version: 8.0.0
APM Agent language and version: JAVA 11
Browser version: Chrome
Original install method (e.g. download page, yum, deb, from source, etc.) and version: Deployed ELK stack with APM server using Helm3 and ECK operator in Azure kubernetes env.
Fresh install or upgraded from other version? Fresh
Is there anything special in your setup? NO
I'm exploring APM for microservices. Servers are running fine in k8s.
Added apm agent to microservice and its able to send logs to apm server but APM is not able to send logs to Elasticsearch.
apm server logs
{"log.level":"info","@timestamp":"2022-04-01T10:08:11.758Z","log.logger":"request","log.origin":{"file.name":"middleware/log_middleware.go","file.line":61},"message":"request accepted","service.name":"apm-server","url.original":"/intake/v2/events","http.request.method":"POST","user_agent.original":"apm-agent-java/1.29.0 (apm-rules 0.0.1-SNAPSHOT)","source.address":"10.244.16.157","http.request.id":"95caa481-eb81-4671-bf28-c556295856a3","event.duration":10012098365,"http.response.status_code":202,"ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-04-01T10:08:12.335Z","log.logger":"modelindexer","log.origin":{"file.name":"modelindexer/indexer.go","file.line":388},"message":"failed to index event (security_exception): action [indices:admin/auto_create] is unauthorized for user [efk-dev-efk-dev-apm-user] with roles [apm_system,eck_apm_user_role_v75,ingest_admin] on indices [metrics-apm.app.apm_sync_data-default], this action is granted by the index privileges [auto_configure,create_index,manage,all]","service.name":"apm-server","ecs.version":"1.6.0"}
failed to index event (security_exception): action [indices:admin/auto_create] is unauthorized for user [efk-dev-efk-dev-apm-user] with roles [apm_system,eck_apm_user_role_v75,ingest_admin] on indices [metrics-apm.app.apm_sync_data-default]
apmserver.yaml
apiVersion: apm.k8s.elastic.co/v1
kind: ApmServer
metadata:
name: {{ .Release.Name }}
namespace: efk-dev
spec:
type: ApmServer
version: {{ .Values.elasticStackVersion }}
count: 1
elasticsearchRef:
name: {{ .Release.Name }}
kibanaRef:
name: {{ .Release.Name }}
podTemplate:
spec:
containers:
- name: apm-server
resources:
limits:
memory: 1Gi
cpu: 1