Filebeat on ECK cannot create an index due to user privilage problemes

[khaled_belgacem (khaled)]

Hello Everyone,

On my way discovering ECK I'm running a filebeat instance on my ECK cluster so I can read from a file "ip.txt" and index it to a specific index in elasticsearch named "vpndetect", I keep getting an error telling me filebeat cannot create a new index due to a lack of privilege, here is the error log:

"reason":"action [indices:admin/auto_create] is unauthorized for user [rs-filebeatvpn-rs-beat-user] with effective roles [beats_admin,eck_beat_es_filebeat_role_v77,ingest_admin,kibana_admin,remote_monitoring_agent] on indices [vpndetect-8.5.0], this action is granted by the index privileges [auto_configure,create_index,manage,all]"}, dropping event!","service.name":"filebeat","ecs.version":"1.6.0"}

Seems to me that when deploying Filebeat on Eck it creates a default user with specific privileges, I tried following this track, see if i can edit this default config and I couldn't find anything.

Any one have an idea of how to resolve this issue or the only solution is to put everything in the same index?

Please find here below my filebeat config (i'm using helm that's why some values are between curly braces)

Thank you.

apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
  name: filebeatvpn-{{.Values.Kubernetes.Namespace}}
  namespace: {{.Values.Kubernetes.Namespace}}
spec:
  type: filebeat
  version: {{.Values.Elastic.STACK_VERSION}}
  elasticsearchRef:
    name: {{.Values.Elastic.ECN}}
  kibanaRef:
    name: {{.Values.Kibana.KiN}}
  config:
    output.elasticsearch.index: "vpndetect-%{[agent.version]}"
    filebeat.inputs:
    - type: filestream
      id: vpn
      paths:
      - /usr/share/filebeat/vpn/ip.txt
    setup:
      template.name: "vpndetect"
      template.pattern: "vpndetect-%{[agent.version]}"
      ilm:
        check_exists: true
        enabled: true
        overwrite: false
        policy_file: /usr/share/filebeat/rsiem-filebeat-lifecycle-policy.json
        policy_name: filebeat-lifecycle-policy
        rollover_alias: filebeat
    http.enabled: true
    http.port: {{.Values.Ports.vpn}}
    http.host: 0.0.0.0
    monitoring.enabled: false
  deployment:
    replicas: 1
    podTemplate:
      metadata:
        labels:
          stack-monitoring.elastic.co/type: beat
        annotations:
          co.elastic.logs/enabled: "true"
      spec:
        nodeName: worker-01
        securityContext:
          runAsUser: 0
        containers:
        - name: filebeat
          ports:
          - containerPort: {{.Values.Ports.vpn}}
            name: monitoring
            protocol: TCP
          volumeMounts:
          - mountPath: usr/share/filebeat/rsiem-filebeat-lifecycle-policy.json
            name: mod
            subPath: rsiem-filebeat-lifecycle-policy.json
          - mountPath: /usr/share/filebeat/vpn
            name: host-mount
        volumes:
        - name: mod
          configMap:
            name: filebeat-{{.Values.Kubernetes.Namespace}}
        - name: host-mount
          hostPath:
            path: /home/rs/vpn