# Filebeat running on ECK cannot create an index

**URL:** <https://discuss.elastic.co/t/filebeat-running-on-eck-cannot-create-an-index/316464>\
**Category:** Elastic Cloud on Kubernetes (ECK)\
**Tags:** elastic-stack-security\
**Created:** [October 12, 2022, 6:54pm UTC](https://discuss.elastic.co/t/filebeat-running-on-eck-cannot-create-an-index/316464 "2022-10-12T18:54:21Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![cloudmatt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cloudmatt/32/111991_2.png) [@cloudmatt](https://discuss.elastic.co/u/cloudmatt)\
**Post date:** [October 12, 2022, 6:54pm UTC](https://discuss.elastic.co/t/filebeat-running-on-eck-cannot-create-an-index/316464/1 "2022-10-12T18:54:21Z")

</div>

I'm trying to wrap my head around running ECK and using filebeat. I've successfully deployed filebeat and I see it grabbing logs from my GCP pub/sub topic but it keeps dropping them because of the following error:

```auto
Private:(*pubsub.Message)(0xc00047e9a0), TimeSeries:false}, Flags:0x1, Cache:publisher.EventCache{m:mapstr.M(nil)}} (status=403): {\"type\":\"security_exception\",\"reason\":\"action [indices:admin/auto_create] is unauthorized for user [elastic-filebeat-beat-user] with roles [beats_admin,eck_beat_es_filebeat_role_v77,ingest_admin,kibana_admin,remote_monitoring_agent] on indices [google-auditlog-2022.10.12], this action is granted by the index privileges [auto_configure,create_index,manage,all]\"}, dropping event!","service.name":"filebeat","ecs.version":"1.6.0"}

```

it's saying filbeat doesn't have the permissions to auto create an index but I don't see anywhere where I can change these permissions or update filebeat to use some other role. Could I get some guidance on how to resolve? Relevant snippet from k8s manifest:

```auto
apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
  name: filebeat
spec:
  type: filebeat
  version: 8.4.3
  elasticsearchRef:
    name: elasticsearch
  kibanaRef:
    name: kibana
  config:
    filebeat.inputs:
    - type: gcp-pubsub
      project_id: mycoolproject
      topic: topic-topic-topic
      subscription.name: topic-sub-sub
      tags: ["google", "google-workspace-auditlog"]
      json.fields_under_root: true
      index: "google-auditlog-%{+yyyy.MM.dd}"

```

I've followed the 'getting started' guide and have the ECK operator and all the CRDs installed as well. The logs from the operator statefulset don't show any errors or anything either.

---

<div class="post-metadata">

**Author:** ![krische](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/krische/32/112302_2.png) [@krische](https://discuss.elastic.co/u/krische)\
**Post date:** [October 19, 2022, 8:52pm UTC](https://discuss.elastic.co/t/filebeat-running-on-eck-cannot-create-an-index/316464/2 "2022-10-19T20:52:48Z")

</div>

I think I'm running into the same issue as you. I want one of my filebeat instances to send data to a different Elasticsearch index. But when I do that, I see the same errors as you.

From what I can read in the ECK documentation, it sounds like ECK is automatically creating a user in Elasticsearch for each beat instance. My guess is that the user generated is probably only given privileges for the default index name, like `filebeat-*`. I can't find any proof of that though or if it's even possible to change that if that is the case.

---

<div class="post-metadata">

**Author:** ![cloudmatt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cloudmatt/32/111991_2.png) [@cloudmatt](https://discuss.elastic.co/u/cloudmatt)\
**Post date:** [October 31, 2022, 4:33pm UTC](https://discuss.elastic.co/t/filebeat-running-on-eck-cannot-create-an-index/316464/3 "2022-10-31T16:33:05Z")

</div>

Yeah - I tried looking through all the docs and couldn't make sense of it. Unfortunate there hasn't been a response to this post ☹

Guess i'll just shove everything into a single index and hope it works for the best 🤷

---

<div class="post-metadata">

**Author:** ![khaled\_belgacem](https://avatars.discourse-cdn.com/v4/letter/k/45deac/32.png) [@khaled\_belgacem](https://discuss.elastic.co/u/khaled_belgacem)\
**Post date:** [November 21, 2022, 1:34pm UTC](https://discuss.elastic.co/t/filebeat-running-on-eck-cannot-create-an-index/316464/5 "2022-11-21T13:34:10Z")

</div>

Hello Matt,

i faced the same probleme 2 days ago and read your topic while looking for a solution, now i just solved it and i thought you may still looking for a way to do this, here's how it worked for me:

in the yaml file for your filebeat under output.elasticsearch and setup.kibana specify the username and the password of an elasticsearch user that you have to create manualy from kibana; give that user the appropriate privillages over the index you need to create by adding a new role, please find here below my config file and a screenshot for a clear understanding

```auto
apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
  name: filebeatvpn-{{.Values.Kubernetes.Namespace}}
  namespace: {{.Values.Kubernetes.Namespace}}
spec:
  type: filebeat
  version: {{.Values.Elastic.STACK_VERSION}}
  elasticsearchRef:
    name: {{.Values.Elastic.ECN}}
  kibanaRef:
    name: {{.Values.Kibana.KiN}}
  config:
    output.elasticsearch:
      index: "vpndetect-%{[agent.version]}"
      username: "username"
      password: "password"
    filebeat.inputs:
    - type: filestream
      id: vpn
      paths:
      - /usr/share/vpnip/ip.txt
    setup:
      kibana:
        username: "username"
        password: "password"
      template.name: "vpndetect"
      template.pattern: "vpndetect-%{[agent.version]}"
      ilm:
        check_exists: true
        enabled: true
        overwrite: false
        policy_file: /usr/share/filebeat/rsiem-filebeat-lifecycle-policy.json
        policy_name: filebeat-lifecycle-policy
        rollover_alias: vpndetect
    http.enabled: true
    http.port: {{.Values.Ports.vpn}}
    http.host: 0.0.0.0
    monitoring.enabled: false
  deployment:
    replicas: 1
    podTemplate:
      metadata:
        labels:
          stack-monitoring.elastic.co/type: beat
        annotations:
          co.elastic.logs/enabled: "true"
      spec:
        nodeName: worker-01
        securityContext:
          runAsUser: 0
        containers:
        - name: filebeat
          ports:
          - containerPort: {{.Values.Ports.vpn}}
            name: monitoring
            protocol: TCP
          volumeMounts:
          - mountPath: usr/share/filebeat/rsiem-filebeat-lifecycle-policy.json
            name: mod
            subPath: rsiem-filebeat-lifecycle-policy.json
          - mountPath: /usr/share/vpnip
            name: host-mount
        volumes:
        - name: mod
          configMap:
            name: filebeat-{{.Values.Kubernetes.Namespace}}
        - name: host-mount
          hostPath:
            path: /home/rs/vpn                                       

```

now here is the role config (a simple "vpndetect\*" and an "all" privillage would have worked i'll change this later):

 ![role](https://us1.discourse-cdn.com/elastic/original/3X/1/d/1dc1fb5fc9acf4da38750f2b79d9ba9f4d547c27.png)

and this is the user:

 ![user](https://us1.discourse-cdn.com/elastic/original/3X/0/c/0ca7c604e64328c2098c092c43d0f22e24284d65.png)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 19, 2022, 1:34pm UTC](https://discuss.elastic.co/t/filebeat-running-on-eck-cannot-create-an-index/316464/6 "2022-12-19T13:34:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.