FIlebeat fails to collect events of a particular pod that is continuously generating logs

[bhavaniprasad_reddy]

I am running filebeat - 7.9.3 to collect log events from a kubernetes cluster using autodiscover.
There are around 5k events got generated with a specific tag in "ne-db-manager" pod but filebeat collected only 500+ records.
Also the filebeat events count stops after 500+ even though the tag count keeps on increasing. Please let me know the debugging steps to find the cause,

filebeat.yml: |-
  filebeat.autodiscover:
    providers:
      - type: kubernetes
        node: ${NODE_NAME}
        tags:
          - "kube-logs"
        templates:
          - condition:
              or:
                - contains:
                    kubernetes.pod.name: "ne-mgmt"
                - contains:
                    kubernetes.pod.name: "list-manager"
                - contains:
                    kubernetes.pod.name: "scheduler-mgmt"
                - contains:
                    kubernetes.pod.name: "sync-ne"
                - contains:
                    kubernetes.pod.name: "file-manager"
                - contains:
                    kubernetes.pod.name: "dash-board"
                - contains:
                    kubernetes.pod.name: "ne-db-manager"
                - contains:
                    kubernetes.pod.name: "config-manager"
                - contains:
                    kubernetes.pod.name: "report-manager"
                - contains:
                    kubernetes.pod.name: "clean-backup"
                - contains:
                    kubernetes.pod.name: "warrior"
                - contains:
                    kubernetes.pod.name: "ne-backup"
                - contains:
                    kubernetes.pod.name: "ne-restore"
            config:
              - type: container
                paths:
                  - "/var/log/containers/*-${data.kubernetes.container.id}.log"
  logging.level: debug
  processors:
    - drop_event:
        when.or:
           - equals:
               kubernetes.namespace: "kube-system"
           - equals:
               kubernetes.namespace: "default"
           - equals:
               kubernetes.namespace: "logging"
  output.logstash:
    hosts: ["logstash-service.logging:5044"]
    index: filebeat
    pretty: true
  setup.template.name: "filebeat"
  setup.template.pattern: "filebeat-*"

I do not see any particular errors from the filebeat logs other than a sequence of stop events.
Please find a portion of log here - filebeat.log - 8b1ddbfe