Hello! I created API key for Filebeat to publish log records to Elasticsearch using this documentation as a reference: Grant privileges and roles needed for publishing | Filebeat Reference [8.6] | Elastic.
But after I deployed Filebeat I noticed that there was an error in its log: action [indices:admin/mapping/auto_put] is unauthorized for API key id ... this action is granted by the index privileges [auto_configure,manage,write,all]. And after I added "write" privilige on the index, it started to publish with no problems. So is there a missing information in the documentation about all necessary permissions or I didn't understand something?
Curious if you can share your filebeat.yml / configuration.
Hello! Of course, here is my Filebeat configuration. But what's interesting is that I've tested it again using a new API key without "write" permission and it worked. So it looks like it didn't work only the first time after I created datastream and ILM using setup role, but currently Filebeat can publish without this premission. But what is strange that deploying Metricbeat I got the same message of the error {\"type\":\"security_exception\",\"reason\":\"action [indices:admin/mapping/auto_put] is unauthorized for API key id [vWCXL4cBDD1EFNIczWAz] of user [373100111], this action is granted by the index privileges [auto_configure,manage,write,all]\"}, dropping event!","service.name":"metricbeat","ecs.version":"1.6.0"} but this time it was shown as "warn" and after I added "write" permission it disappeared.
apiVersion: v1
kind: ServiceAccount
metadata:
name: filebeat
namespace: beats
labels:
k8s-app: filebeat
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: filebeat
labels:
k8s-app: filebeat
rules:
- apiGroups: [""] # "" indicates the core API group
resources:
- namespaces
- pods
- nodes
verbs:
- get
- watch
- list
- apiGroups: ["apps"]
resources:
- replicasets
verbs: ["get", "list", "watch"]
- apiGroups: ["batch"]
resources:
- jobs
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: filebeat
# should be the namespace where filebeat is running
namespace: beats
labels:
k8s-app: filebeat
rules:
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs: ["get", "create", "update"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: filebeat-kubeadm-config
namespace: beats
labels:
k8s-app: filebeat
rules:
- apiGroups: [""]
resources:
- configmaps
resourceNames:
- kubeadm-config
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: filebeat
subjects:
- kind: ServiceAccount
name: filebeat
namespace: beats
roleRef:
kind: ClusterRole
name: filebeat
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: filebeat
namespace: beats
subjects:
- kind: ServiceAccount
name: filebeat
namespace: beats
roleRef:
kind: Role
name: filebeat
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: filebeat-kubeadm-config
namespace: beats
subjects:
- kind: ServiceAccount
name: filebeat
namespace: beats
roleRef:
kind: Role
name: filebeat-kubeadm-config
apiGroup: rbac.authorization.k8s.io
---
apiVersion: v1
kind: ConfigMap
metadata:
name: filebeat-config
namespace: beats
labels:
k8s-app: filebeat
data:
filebeat.yml: |-
filebeat.autodiscover:
providers:
- type: kubernetes
node: ${NODE_NAME}
hints.enabled: true
hints.default_config:
type: container
paths:
- /var/log/containers/*${data.kubernetes.container.id}.log
multiline.type: pattern
multiline.pattern: '^[[:space:]]'
multiline.negate: false
multiline.match: after
processors:
- add_cloud_metadata:
- add_host_metadata:
- drop_event:
when:
not:
contains:
kubernetes.namespace: my-app
setup.ilm.check_exists: false
cloud.id: ${ELASTIC_CLOUD_ID}
# cloud.auth: ${ELASTIC_CLOUD_AUTH}
output.elasticsearch:
api_key: ${ELASTIC_API_KEY}
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: filebeat
namespace: beats
labels:
k8s-app: filebeat
spec:
selector:
matchLabels:
k8s-app: filebeat
template:
metadata:
labels:
k8s-app: filebeat
spec:
serviceAccountName: filebeat
terminationGracePeriodSeconds: 30
hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet
containers:
- name: filebeat
image: docker.elastic.co/beats/filebeat:8.6.2
args: ["-c", "/etc/filebeat.yml", "-e"]
env:
- name: ELASTIC_CLOUD_ID
value: my-cloud-id
- name: ELASTIC_API_KEY
value: my-key
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
securityContext:
runAsUser: 0
# If using Red Hat OpenShift uncomment this:
#privileged: true
resources:
limits:
memory: 1Gi
requests:
cpu: 100m
memory: 100Mi
volumeMounts:
- name: config
mountPath: /etc/filebeat.yml
readOnly: true
subPath: filebeat.yml
- name: data
mountPath: /usr/share/filebeat/data
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
- name: varlog
mountPath: /var/log
readOnly: true
volumes:
- name: config
configMap:
defaultMode: 0640
name: filebeat-config
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
- name: varlog
hostPath:
path: /var/log
# data folder stores a registry of read status for all files, so we don't send everything again on a Filebeat pod restart
- name: data
hostPath:
# When filebeat runs as non-root user, this directory needs to be writable by group (g+w).
path: /var/lib/filebeat-data
type: DirectoryOrCreate
---
Hmmm thinking the docs may be out of date but not sure...
Your config looks good... was thinking there was an index vs data stream which still may be under the covers... But looks like a data stream
Data Streams need create
Indices need write
Subtle Difference
Strange. Actually yesterday I noticed that the Filebeat that I left without "write" permission was receiving warns in it log again and the index in the Elastic was empty. After I gave "write" everything returned to work fine. So in my case for some reason, event hough I'm using datastreams, without "write" Filebeat is not able to publish to Elasticsearch.
{"log.level":"warn","@timestamp":"2023-04-03T18:19:58.738Z","log.logger":"elasticsearch","log.origin":{"file.name":"elasticsearch/client.go","file.line":429},"message":"Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Date(2023, time.April, 3, 15, 33, 3, 625287975, time.UTC), Meta:null .............. Offset:1192332, Timestamp:time.Date(2023, time.April, 3, 17, 58, 17, 387060900, time.Local), TTL:-1, Type:\"container\", Meta:map[string]string(nil), FileStateOS:file.StateOS{Inode:0x470f95, Device:0x801}, IdentifierName:\"native\"}, TimeSeries:false}, Flags:0x1, Cache:publisher.EventCache{m:mapstr.M(nil)}} (status=403): {\"type\":\"security_exception\",\"reason\":\"action [indices:admin/mapping/auto_put] is unauthorized for API key id [Zir2NIcBcENJVIM27ua3] of user [373100111], this action is granted by the index privileges [auto_configure,manage,write,all]\"}, dropping event!","service.name":"filebeat","ecs.version":"1.6.0"}
So I assume you are using 8.6 because you referenced the docs.
Hmm interesting... I have not tested all the permutations recently... if I get a chance at some point I will write vs create is a very fine line.. perhaps something changed and the docs are out of date.
Also I do not see the actual API KEY creation so hard for me to tell...
AND see there has been a bit of an update see here
So I was thinking of create as the more refined create_doc
create
Privilege to index documents.
Note: This privilege does not restrict the index operation to the creation of documents but instead restricts API use to the index API. The index API allows a user to overwrite a previously indexed document. See thecreate_docprivilege for an alternative.
create_doc
Privilege to index documents. It does not grant the permission to update or overwrite existing documents.
Note: This privilege relies on theop_typeof indexing requests (Index and Bulk). When ingesting documents as a user who has thecreate_docprivilege (and no higher privilege such asindexorwrite), you must ensure that op_type is set to create through one of the following:
- Explicitly setting the
op_typein the index or bulk APIs- Using the
_createendpoint for the index API- Creating a document with an auto-generated
_id
write
Privilege to perform all write operations to documents, which includes the permission to index, update, and delete documents as well as performing bulk operations, while also allowing to dynamically update the index mapping.
So seems that create would work for both now. There is some subtleties with the create_doc and the op_type
But I am unclear on what roles your actual API KEY was and what version you are using...
AND it is always possible that the error message is a bit out of date.
I am not sure which combination it is...
One thing just in case, and I am sure this is probably not it... but just in case an API key can not have greater permission than the user that creates it... so example if the user creating the API does not have write permission even if you add write permission in the API KEY Create API Call you will see it but it will not be honored (which makes sense otherwise you could just escalate the privilege) Probably not the problem but I have seen that...
1 Like
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.