We are currently using a filebeat daemon set with the "old" container input, both version 7.17.x
we want to start migrating to the newer approach of hint based log collection
for this we need the old filebeat daemon set to run alongside the new autodiscover deployment
for hint based it's easy to only collect logs from pods that use the annotations since we can turn the default configuration off
The problem is we now want the old filebeat to filter out any pod/container that is handled by the auto discover
Since add_kubernetes_metadata does not add pod annotations we cannot use drop_event with a condition based on annotations
Other than manually adding a label in addition to the annotations, is there a solution for running the 2 types of inputs at the same time?
When you say "not included in the available configuration options..." you mean it's undocumented?
from the docs i see we can add resource annotations (nodes,namespaces) but not for pods.
Ill try your suggestion anyway, currently we decided to add a label and filter by that
Yes it is undocumented. Keep in mind that this option is not part of the add_resource_metadata. There you can specify which metadata of nodes and namespaces you want the events to be enriched with.
It has to be added on the higher level of the processor configuration.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.