I'm migrating from quite an old ES+fluentbit configuration (logging solution for a small kubernetes cluster).
And this is quite simple yet inefficient (?) config I came up with (this file is generated by ECK using the Beat CRD):
# cat /etc/beat.yml
filebeat:
autodiscover:
providers:
- add_resource_metadata:
cronjob: true
deployment: true
hints:
default_config:
file_identity:
fingerprint: null
id: kubernetes-container-logs-${data.kubernetes.pod.name}-${data.kubernetes.container.id}
parsers:
- container: null
paths:
- /var/log/containers/*${data.kubernetes.container.id}.log
prospector:
scanner:
fingerprint:
enabled: true
symlinks: true
type: filestream
enabled: true
node: ${NODE_NAME}
type: kubernetes
inputs:
- fields:
log_source: dmesg
fields_under_root: true
id: dmesg
include_matches:
match:
- _TRANSPORT=kernel
paths:
- /var/log/journal
seek: since
since: -24h
type: journald
- fields:
log_source: journald
fields_under_root: true
id: systemd-services
paths:
- /var/log/journal
seek: since
since: -24h
type: journald
units:
- containerd.service
- kubelet.service
output:
elasticsearch:
hosts:
- http://elasticsearch-es-http.<redacted>.svc:9200
password: <redacted>
ssl:
certificate_authorities:
- /mnt/elastic-internal/elasticsearch-certs/ca.crt
username: <redacted>-container-logs-beat-user
On a node with ~200 pods (and ~430 containers) the filebeat process consumes around 0.7-0.8 CPU constantly. While fluentbit is at ~0.01 on the same machine.
Question: is filebeat really so inefficient per-se, or I have made some obviously terrible mistake configuring it?