I have web app service that run in go and produce a log that I stored inside /var/logs/app/app.log
. This service is running inside a Kubernetes pods. And I have another pods that running Filebeat to listen to logs produced by app service. But my Filebeat doesn't seems can listen to the logs produced by apps.
This is my config map for filebeat deployment
apiVersion: v1
kind: ConfigMap
metadata:
name: filebeat-config
labels:
k8s-app: filebeat
data:
filebeat.yml: |-
filebeat.autodiscover:
providers:
- type: kubernetes
templates:
- config:
- type: container
paths:
- /var/logs/app/*.log
processors:
- decode_json_fields:
fields: ['message']
target: json
- rename:
fields:
- from: "url"
to: "url_data"
ignore_missing: false
fail_on_error: true
cloud.id: ${CLOUD_ID}
cloud.auth : ${CLOUD_AUTH}
And this is my deployment config
apiVersion: apps/v1
kind: Deployment
metadata:
name: filebeat
labels:
k8s-app: filebeat
spec:
selector:
matchLabels:
k8s-app: filebeat
template:
metadata:
labels:
k8s-app: filebeat
spec:
serviceAccountName: filebeat
terminationGracePeriodSeconds: 30
hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet
containers:
- name: filebeat
image: docker.elastic.co/beats/filebeat:8.8.1
args: [
"-c", "/etc/filebeat.yml",
"-e",
]
env:
- name: CLOUD_ID
valueFrom:
secretKeyRef:
name: filebeat-secret
key: cloud.id
- name: CLOUD_AUTH
valueFrom:
secretKeyRef:
name: filebeat-secret
key: cloud.auth
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
securityContext:
runAsUser: 0
resources:
limits:
memory: 200Mi
requests:
cpu: 100m
memory: 100Mi
volumeMounts:
- name: config
mountPath: /etc/filebeat.yml
readOnly: true
subPath: filebeat.yml
volumes:
- name: config
configMap:
defaultMode: 0640
name: filebeat-config
I expect filebeat can listen custom log instead of docker logs. Where did I go wrong that filebeat can't listen to my app logs.