# How to add podip on add\_kubernetes\_metadata processor?

**URL:** <https://discuss.elastic.co/t/how-to-add-podip-on-add-kubernetes-metadata-processor/269006>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [April 1, 2021, 9:25am UTC](https://discuss.elastic.co/t/how-to-add-podip-on-add-kubernetes-metadata-processor/269006 "2021-04-01T09:25:30Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![dadayoo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadayoo/32/86458_2.png) [@dadayoo](https://discuss.elastic.co/u/dadayoo)\
**Post date:** [April 1, 2021, 9:25am UTC](https://discuss.elastic.co/t/how-to-add-podip-on-add-kubernetes-metadata-processor/269006/1 "2021-04-01T09:25:30Z")

</div>

Hi I'm running filebeat daemonset on my k8s cluster  
have try different config scenario but do not get the pod ip data  
here is my config

```auto
  processors:
    - add_kubernetes_metadata:
        default_indexers.enabled: true
        default_matchers.enabled: true
        indexers:
          - pod_uid:

```

am i miss something?

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [April 1, 2021, 9:51am UTC](https://discuss.elastic.co/t/how-to-add-podip-on-add-kubernetes-metadata-processor/269006/2 "2021-04-01T09:51:54Z")

</div>

Hi @dadayoo, welcome to discuss 🙂

What filebeat configuration are you using? Take a look to the reference manifests provided in beats: [beats/filebeat-kubernetes.yaml at v7.12.0 · elastic/beats · GitHub](https://github.com/elastic/beats/blob/v7.12.0/deploy/kubernetes/filebeat-kubernetes.yaml)

Specifically to the `add_kubernetes_metadata` example. Something like this should work (if your logs are under `/var/log/containers`:

```auto
        - add_kubernetes_metadata:
            host: ${NODE_NAME}
            matchers:
            - logs_path:
                logs_path: "/var/log/containers/"

```

Or alternativelly you can use autodiscover as commented in the same reference config. Autodiscover already fills in the kubernetes metadata and then `add_kubernetes_metadata` is not needed.

---

<div class="post-metadata">

**Author:** ![dadayoo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadayoo/32/86458_2.png) [@dadayoo](https://discuss.elastic.co/u/dadayoo)\
**Post date:** [April 1, 2021, 11:45am UTC](https://discuss.elastic.co/t/how-to-add-podip-on-add-kubernetes-metadata-processor/269006/3 "2021-04-01T11:45:40Z")

</div>

hi @jsoriano thanks for your replay  
here is my filebeat config

```auto
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/lib/kubelet/pods/*/volumes/kubernetes.io~empty-dir/logs/applogs/*/tomcat_stdout.log

  processors:
  - add_kubernetes_metadata:
      host: ${NODE_NAME}
      matchers:
      - logs_path:
          logs_path: "/var/log/containers/"

output.console:
  pretty: true

```

here is console output

```auto
{
  "@timestamp": "2021-04-01T11:38:37.351Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "_doc",
    "version": "7.12.0"
  },
  "ecs": {
    "version": "1.8.0"
  },
  "host": {
    "name": "filebeat-new-6d87884ccf-797vm"
  },
  "agent": {
    "ephemeral_id": "519086ef-c4d0-4b12-84a7-5feea55af52c",
    "id": "8fca3548-29b3-4510-883b-9fba7cd82cce",
    "name": "filebeat-new-6d87884ccf-797vm",
    "type": "filebeat",
    "version": "7.12.0",
    "hostname": "filebeat-new-6d87884ccf-797vm"
  },
  "log": {
    "offset": 27567571,
    "file": {
      "path": "/var/lib/kubelet/pods/e5e497e1-8bdb-45b9-9141-c3ca2dce8eb8/volumes/kubernetes.io~empty-dir/logs/applogs/resource-manage/tomcat_stdout.log"
    }
  },
  "message": "application stdout message",
  "input": {
    "type": "log"
  }
}

```

still didn't get podip here 😪

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [April 1, 2021, 12:03pm UTC](https://discuss.elastic.co/t/how-to-add-podip-on-add-kubernetes-metadata-processor/269006/4 "2021-04-01T12:03:43Z")

</div>

Ok, I think the problem is with the path, I guess that these are not logs from stdout/stdin, but from a mounted volume, and is under the pods path, not the containers path.

Try something like the example in these docs to match with the pod logs path: [Add Kubernetes metadata | Filebeat Reference [7.12] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/7.12/add-kubernetes-metadata.html#_logs_path)

```auto
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/lib/kubelet/pods/*/volumes/kubernetes.io~empty-dir/logs/applogs/*/tomcat_stdout.log

  processors:
  - add_kubernetes_metadata:
      host: ${NODE_NAME}
      default_indexers.enabled: false
      default_matchers.enabled: false
      indexers:
        - pod_uid:
      matchers:
        - logs_path:
            logs_path: '/var/lib/kubelet/pods'
            resource_type: 'pod'

```

Or you can try using [autodiscover](https://www.elastic.co/guide/en/beats/filebeat/7.12/configuration-autodiscover.html#_kubernetes) instead of `filebeat.inputs`, that wouldn't need `add_kubernetes_metadata`. It would be something like this (not tested, review the condition and the path):

```auto
filebeat.autodiscover:
  providers:
    - type: kubernetes
      templates:
        - condition:
            contains:
              kubernetes.pod.name: tomcat
          config:
            - type: log
              paths:
                - /var/lib/kubelet/pods/${kubernetes.pod.uid}*/volumes/kubernetes.io~empty-dir/logs/applogs/*/tomcat_stdout.log

```

---

<div class="post-metadata">

**Author:** ![dadayoo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadayoo/32/86458_2.png) [@dadayoo](https://discuss.elastic.co/u/dadayoo)\
**Post date:** [April 1, 2021, 12:41pm UTC](https://discuss.elastic.co/t/how-to-add-podip-on-add-kubernetes-metadata-processor/269006/5 "2021-04-01T12:41:57Z")

</div>

yes, file beat collect logs from Hostpath which is kubernetes emptyDir mount point  
inner pod path:

```auto
/home/sclogs/logs/applogs/myapp/tomcat_stdout.log

```

Host path:

```auto
/var/lib/kubelet/pods/*/volumes/kubernetes.io~empty-dir/logs/applogs/*/tomcat_stdout.log

```

I have try

```auto
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/lib/kubelet/pods/*/volumes/kubernetes.io~empty-dir/logs/applogs/*/tomcat_stdout.log

  processors:
  - add_kubernetes_metadata:
      host: ${NODE_NAME}
      default_indexers.enabled: false
      default_matchers.enabled: false
      indexers:
        - pod_uid:
      matchers:
        - logs_path:
            logs_path: '/var/lib/kubelet/pods'
            resource_type: 'pod'
     
output.console:
  pretty: true

```

console output:

```auto
{
  "@timestamp": "2021-04-01T12:21:03.795Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "_doc",
    "version": "7.12.0"
  },
  "message": "here is application stdout message",
  "input": {
    "type": "log"
  },
  "kubernetes": {
    "labels": {
      "app": "workapp",
      "pod-template-hash": "645bc8fb7b",
      "szone": "sc"
    },
    "node": {
      "name": "node.172.28.131.146",
      "uid": "5f187da8-4650-4fcc-b4ea-61dccfc1410a",
      "labels": {
        "kubernetes_io/os": "linux",
        "beta_kubernetes_io/arch": "amd64",
        "beta_kubernetes_io/os": "linux"
      },
      "hostname": "node.172.28.131.146"
    },
    "namespace_uid": "da7ac4c7-f733-4728-988e-7a5ba6e2dedc",
    "pod": {
      "name": "workapp-645bc8fb7b-qqf5p",
      "uid": "961e07e0-11b3-40c8-92ea-0be804d5d949"
    },
    "namespace": "sc",
    "replicaset": {
      "name": "workapp-645bc8fb7b"
    }
  },
  "ecs": {
    "version": "1.8.0"
  },
  "host": {
    "name": "filebeat-new-6d87884ccf-whszl"
  },
  "agent": {
    "hostname": "filebeat-new-6d87884ccf-whszl",
    "ephemeral_id": "f9bf1229-039b-4921-b4ea-933133286903",
    "id": "d8fcd0ac-5522-4319-9425-289bb5ddf13d",
    "name": "filebeat-new-6d87884ccf-whszl",
    "type": "filebeat",
    "version": "7.12.0"
  },
  "log": {
    "offset": 1944254,
    "file": {
      "path": "/var/lib/kubelet/pods/961e07e0-11b3-40c8-92ea-0be804d5d949/volumes/kubernetes.io~empty-dir/logs/applogs/worknum/tomcat_stdout.log"
    }
  }
}

```

more filed get ,but still no podip  
I have try autodicover, but it get "Non-zero metric"

```auto
filebeat.autodiscover:
  providers:
    - type: kubernetes
      templates:
        - condition:
            contains:
              kubernetes.pod.name: tomcat
          config:
            - type: log
              paths:
                - /var/lib/kubelet/pods/${kubernetes.pod.uid}*/volumes/kubernetes.io~empty-dir/logs/applogs/*/tomcat_stdout.log

output.console:
  pretty: true

```

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [April 1, 2021, 2:07pm UTC](https://discuss.elastic.co/t/how-to-add-podip-on-add-kubernetes-metadata-processor/269006/6 "2021-04-01T14:07:53Z")

</div>

Umm, I thought that pod IP was included in metadata, but not, it seems it is only included in the data (as `data.host`) that can be used in autodiscover templates. When collecting metrics, the endpoint uses to be also available in the event, but I see now that it is not included in log events.

I have created an issue to follow on this: [Add pod IP to kubernetes metadata · Issue #24902 · elastic/beats · GitHub](https://github.com/elastic/beats/issues/24902)

As a possible workaround, as this IP is available in autodiscover templates, if you get a working configuration maybe you can fill the field with a processor, something like this:

```auto
filebeat.autodiscover:
  providers:
    - type: kubernetes
      templates:
        - condition:
            contains:
              kubernetes.pod.name: workapp
          config:
            - type: log
              paths:
                - /var/lib/kubelet/pods/${data.kubernetes.pod.uid}/volumes/kubernetes.io~empty-dir/logs/applogs/*/tomcat_stdout.log
              processors:
              - add_fields:
                  target: kubernetes.pod
                  fields:
                    ip: "${data.host}"

output.console:
  pretty: true

```

Notice that I have changed some things:

- Adapted the condition to match your pod, review it if you want to match a different set of pods.
- Fixed variable used in path in config, they have to be prefixed by `data.`.
- Added `add_fields` processor to add `kubernetes.pod.ip` with the value of `data.host`.

---

<div class="post-metadata">

**Author:** ![dadayoo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadayoo/32/86458_2.png) [@dadayoo](https://discuss.elastic.co/u/dadayoo)\
**Post date:** [April 2, 2021, 7:45am UTC](https://discuss.elastic.co/t/how-to-add-podip-on-add-kubernetes-metadata-processor/269006/7 "2021-04-02T07:45:05Z")

</div>

@jsoriano  
Thanks for help!  
I turn to autodiscover and it works! finally got podip 😋 👍 👍  
hope I can get podip from events metadata on next version of filebeat 😜

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 30, 2021, 9:45am UTC](https://discuss.elastic.co/t/how-to-add-podip-on-add-kubernetes-metadata-processor/269006/8 "2021-04-30T09:45:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.