Is there is method to collect multiple types of elasticsearch logs when elasticsearchs and filebeats are deployed with ECK

I am using ECK1.6 to deploy Elasticsearch cluster 7.15.2 on k8s,and then using ECK to deploy filebeat to collect Elasticsearch server log, gc log, slowlog, deprecation log. There is an error as same as this issue Filebeat registry collisions when an integration is used twice #22054 when I using autodiscover feature and configure a template for echo types of log (server log, gc log, slowlog, deprecation log). While everything is OK when I comment out other three template and collect server logs only.
So, Is there is method to collect multiple types of Elasticsearch logs when filebeat and Elasticsearch are deployed with ECK?
My filebeat manifest is modified from filebeat_autodiscover_by_metadata.yaml

Not sure if it helps, but worth trying to add IDs:

      - condition.equals.kubernetes.namespace: log-namespace
        config:
        - paths: ["/var/log/containers/*${data.kubernetes.container.id}.log"]
          type: container
          id: "first"
      - condition.equals.kubernetes.labels.log-label: "true"
        config:
        - paths: ["/var/log/containers/*${data.kubernetes.container.id}.log"]
          type: container
          id: "second"

Thank you.
The same error is returned with "id" set. My filebeat.yaml is like this below:

      - condition
          equals:
            kubernetes:
              namespace: elastic-system
              labels:
                common_k8s_elastic_co/type: elasticsearch
        config:
        - module: elasticsearch
          slowlog:
            input:
              id: "slowlog"
              paths: ["/var/log/containers/*${data.kubernetes.container.id}.log"]
              type: container
              include_lines:
              - "type":\s"(index_search_slowlog|index_indexing_slowlog)"
      - condition
          equals:
            kubernetes:
              namespace: elastic-system
              labels:
                common_k8s_elastic_co/type: elasticsearch
        config:
        - module: elasticsearch
          server:
            input:
              id: "server"
              paths: ["/var/log/containers/*${data.kubernetes.container.id}.log"]
              type: container
              include_lines:
              - "type":\s"server"}

I find a perfect solution, it helped.

The solution is from this post:
https://xeraa.net/blog/2020_filebeat-modules-with-docker-kubernetes/#planD

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.