Filebeat processor add_kubernetes_metadata is not working with input type log

Elastic Stack Beats
,
1

We are using filebeat v 7.3 over Kubernetes. Along with pod's logs, we also want to push host logs to elasticsearch. We have mounted "/var/log/messages" of host towards the filebeat pod. The logs are successfully being pushed to elasticsearch but kubernetes metadata is not being added to logs.
Here is my fileabeat input configuration-

- type: docker
  containers.ids:
  - "*"
  multiline.pattern: '^DEBUG'
  multiline.negate: true
  multiline.match: 'after'
  processors:
    - add_kubernetes_metadata:
        in_cluster: true
- type: log
  paths:
    - /var/log/host/messages
  processors:
    - add_kubernetes_metadata:
        in_cluster: true

For the first input type docker, we are getting the Kubernetes metadata but for the second type log, we are not getting any metadata.

1 Like
2

Hi @Himanshu_Rajput,

I'm not sure which metadata would be expected for the second block. If you are expecting host metadata, I would suggest add_host_metadata

I think filebeat looks for the pod/container information based on log path, which makes /var/log/host/messages not suitable for adding kubernetes metadata.

3

@pmercado After adding add_host_metdata we only get the host info such as IP and hostname which is actually of filebeat pod, not of actual host.
My only concern is that-
Can we use add_kubernetes_metadata with input type log ?

2 Likes
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.