Hey @wajika,
In your configuration you are using a path with a wildcard that would match all the containers in the node. So every configuration generated, for every pod, will try to harvest any file. You need to setup autodiscover in a way that it generates an specific configuration for each container. Also, when using autodiscover, you don't need to use add_kubernetes_metadata, events should be already enriched by the autodiscover provider.
I think that something like this would work for you:
filebeat.autodiscover:
providers:
- type: kubernetes
node: ${NODE_NAME}
templates:
- condition:
or:
- equals:
kubernetes.namespace: "front"
- equals:
kubernetes.namespace: "back"
config:
- type: container
paths:
- "/var/log/containers/*-${data.kubernetes.container.id}.log"
setup.template.enabled: false
processors:
- add_cloud_metadata:
- add_host_metadata:
output.elasticsearch:
hosts: ${ELASTICSEARCH_HOST}
Other option could be to use hints-based autodiscover, that would allow you to enable collection of logs per namespace using annotations in the namespaces themselves. See my comment about that on this topic: Collect logs for specific containers or namespace in Openshift/Kubernetes