# Gradual migration from container input to kubernetes autodiscover

**URL:** <https://discuss.elastic.co/t/gradual-migration-from-container-input-to-kubernetes-autodiscover/325979>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [February 20, 2023, 1:20pm UTC](https://discuss.elastic.co/t/gradual-migration-from-container-input-to-kubernetes-autodiscover/325979 "2023-02-20T13:20:36Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![OranShuster](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/oranshuster/32/81779_2.png) [@OranShuster](https://discuss.elastic.co/u/OranShuster)\
**Post date:** [February 20, 2023, 1:20pm UTC](https://discuss.elastic.co/t/gradual-migration-from-container-input-to-kubernetes-autodiscover/325979/1 "2023-02-20T13:20:36Z")

</div>

We are currently using a filebeat daemon set with the "old" container input, both version 7.17.x  
we want to start migrating to the newer approach of hint based log collection  
for this we need the old filebeat daemon set to run alongside the new autodiscover deployment  
for hint based it's easy to only collect logs from pods that use the annotations since we can turn the default configuration off  
The problem is we now want the old filebeat to filter out any pod/container that is handled by the auto discover  
Since `add_kubernetes_metadata` does not add pod annotations we cannot use drop\_event with a condition based on annotations

Other than manually adding a label in addition to the annotations, is there a solution for running the 2 types of inputs at the same time?

---

<div class="post-metadata">

**Author:** ![Michalis\_Katsoulis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michalis_katsoulis/32/93500_2.png) [@Michalis\_Katsoulis](https://discuss.elastic.co/u/Michalis_Katsoulis)\
**Post date:** [February 28, 2023, 3:29pm UTC](https://discuss.elastic.co/t/gradual-migration-from-container-input-to-kubernetes-autodiscover/325979/2 "2023-02-28T15:29:41Z")

</div>

Hi @OranShuster ,

I can see that this is not included in the available configuration options of `add_kubernetes_metadata` processor ( [Add Kubernetes metadata | Filebeat Reference [7.17] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/7.17/add-kubernetes-metadata.html)) but you can add pod annotations.

The way to do it is to use the `include_annotations` setting.  
Example:

```auto
  filebeat.yml: |-
    filebeat.inputs:
    - type: container
      paths:
        - /var/log/containers/*.log
      processors:
        - add_kubernetes_metadata:
            host: ${NODE_NAME}
            include_annotations: ["app"]
            matchers:
            - logs_path:
                logs_path: "/var/log/containers/"

```

---

<div class="post-metadata">

**Author:** ![OranShuster](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/oranshuster/32/81779_2.png) [@OranShuster](https://discuss.elastic.co/u/OranShuster)\
**Post date:** [February 28, 2023, 4:32pm UTC](https://discuss.elastic.co/t/gradual-migration-from-container-input-to-kubernetes-autodiscover/325979/3 "2023-02-28T16:32:40Z")

</div>

When you say "not included in the available configuration options..." you mean it's undocumented?  
from the docs i see we can add resource annotations (nodes,namespaces) but not for pods.  
Ill try your suggestion anyway, currently we decided to add a label and filter by that

---

<div class="post-metadata">

**Author:** ![Michalis\_Katsoulis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michalis_katsoulis/32/93500_2.png) [@Michalis\_Katsoulis](https://discuss.elastic.co/u/Michalis_Katsoulis)\
**Post date:** [March 1, 2023, 9:14am UTC](https://discuss.elastic.co/t/gradual-migration-from-container-input-to-kubernetes-autodiscover/325979/4 "2023-03-01T09:14:11Z")

</div>

Yes it is undocumented. Keep in mind that this option is not part of the `add_resource_metadata`. There you can specify which metadata of nodes and namespaces you want the events to be enriched with.  
It has to be added on the higher level of the processor configuration.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 29, 2023, 11:14am UTC](https://discuss.elastic.co/t/gradual-migration-from-container-input-to-kubernetes-autodiscover/325979/5 "2023-03-29T11:14:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.