Logs are missing kubernetes metadata when using filebeat >= 8.9.0

[gparks (Gareth)]

I'm running filebeat as a daemonset in kubernetes, it was previously on 8.8.2 but when upgraded to 8.9.0 logs from the first input stop including the kubernetes metadata but logs from the ingress-nginx input continue to include the metadata.

I confirmed by downgrading back to 8.8.2 that the metadata returns. I've tested all versions up to 8.10.1 (this being the version I initially upgraded to) and they all exhibit the same problem.

I've read the release notes for 8.9.0 and can't find anything obviously related to it

filebeat.inputs:
- type: container
  paths:
    - /var/log/containers/*.log
  processors:
  - add_kubernetes_metadata:
      host: ${NODE_NAME}
      matchers:
      - logs_path:
          logs_path: "/var/log/containers/"
  fields_under_root: true
  exclude_files:
   - "/var/log/containers/nginx-ingress.*log"
   - "/var/log/containers/ingress-nginx.*log"
  fields:
    log_type: system
- type: container
  paths:
    - /var/log/containers/nginx-ingress*.log
    - /var/log/containers/ingress-nginx*.log
  processors:
  - add_kubernetes_metadata:
      host: ${NODE_NAME}
      matchers:
      - logs_path:
          logs_path: "/var/log/containers/"
  fields_under_root: true
  fields:
    log_type: nginx
    kubernetes_namespace: ${POD_NAMESPACE}
    role: ingress-nginx
  close_renamed: true

[GauravBhinda]

hey @gparks , I'm encountering the same problem. Are you able to solve it yet?