# Filebeat unable obtain kubernetes metadata
**URL:** https://discuss.elastic.co/t/filebeat-unable-obtain-kubernetes-metadata/245741
**Category:** Beats
**Tags:** docker, filebeat
**Created:** [August 20, 2020, 10:20am UTC](https://discuss.elastic.co/t/filebeat-unable-obtain-kubernetes-metadata/245741 "2020-08-20T10:20:36Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@dorinand](https://discuss.elastic.co/u/dorinand)
#### Post date: [August 20, 2020, 10:20am UTC](https://discuss.elastic.co/t/filebeat-unable-obtain-kubernetes-metadata/245741/1 "2020-08-20T10:20:37Z")
I am running filebeat in my kubernetes cluster. I used [this](https://github.com/elastic/helm-charts) helm chart to deploy filebeat, logstash and elasticsearch. This is my filebeat configuration:
```
filebeatConfig:
filebeat.yml: |
filebeat.inputs:
- type: container
exclude_files:
- '.*elk.*'
- '.*elasticsearch.*'
paths:
- /var/log/containers/*.log
processors:
- add_kubernetes_metadata:
host: ${NODE_NAME}
matchers:
- logs_path:
logs_path: "/var/log/containers/"
output.logstash:
worker: 3
host: '${NODE_NAME}'
hosts: 'elk-logstash-logstash:5000'
```
I dont want to process logs from `elasticsearch` and `elk` deployments so I am ignoring them. In logstash I am writeing data into index name: `[kubernetes][pod][name]`. If pod name missing, I am writing it to stdout. Unfortunatelly, logs from some containers missing this metadata. The container is alive, Container ID is the same as in the name of log file. So for example, I have prometheus logs but none of them has pod name in messages from filebeat. I can see, that logs missing whole `kubernetes` section. This is my logstash configuration:
```
logstashPipeline:
logstash.conf: |
input {
beats {
port => 5000
}
}
filter {
}
output {
if ![kubernetes][pod][name] {
stdout { codec => rubydebug { metadata => true }}
} else {
elasticsearch {
hosts => ["http://elasticsearch-data:9200"]
index => "%{[kubernetes][pod][name]}"
}
}
}
```
I am using helm 3.
**UPDATE 1:**
What is interesting, from some containers, e.g. `grafana` container, some logs contains kubernetes metadata, but some not...
---
### Author:  [@jsoriano](https://discuss.elastic.co/u/jsoriano)
#### Post date: [August 26, 2020, 8:50pm UTC](https://discuss.elastic.co/t/filebeat-unable-obtain-kubernetes-metadata/245741/2 "2020-08-26T20:50:11Z")
Hey @dorinand,
What version of filebeat are you using? Do you see any error in logs related to `add_kubernetes_metadata`?
> [@dorinand](#):
>
> What is interesting, from some containers, e.g. `grafana` container, some logs contains kubernetes metadata, but some not...
I wonder if there may be multiple inputs trying to read the same logs, with different configurations or from different paths. Do you see duplicated logs? Do all the logs you see for the same container have the same `log.file.path` in stored events?
By the way, have you considered to send the logs directly to Elasticsearch, and use default indexes? Is there any reason why you are using Logstash, and one index per pod?
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 23, 2020, 10:50pm UTC](https://discuss.elastic.co/t/filebeat-unable-obtain-kubernetes-metadata/245741/3 "2020-09-23T22:50:11Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.