I'm running filebeat as a daemonset in kubernetes, it was previously on 8.8.2 but when upgraded to 8.9.0 logs from the first input stop including the kubernetes metadata but logs from the ingress-nginx input continue to include the metadata.
I confirmed by downgrading back to 8.8.2 that the metadata returns. I've tested all versions up to 8.10.1 (this being the version I initially upgraded to) and they all exhibit the same problem.
I've read the release notes for 8.9.0 and can't find anything obviously related to it
filebeat.inputs:
- type: container
paths:
- /var/log/containers/*.log
processors:
- add_kubernetes_metadata:
host: ${NODE_NAME}
matchers:
- logs_path:
logs_path: "/var/log/containers/"
fields_under_root: true
exclude_files:
- "/var/log/containers/nginx-ingress.*log"
- "/var/log/containers/ingress-nginx.*log"
fields:
log_type: system
- type: container
paths:
- /var/log/containers/nginx-ingress*.log
- /var/log/containers/ingress-nginx*.log
processors:
- add_kubernetes_metadata:
host: ${NODE_NAME}
matchers:
- logs_path:
logs_path: "/var/log/containers/"
fields_under_root: true
fields:
log_type: nginx
kubernetes_namespace: ${POD_NAMESPACE}
role: ingress-nginx
close_renamed: true