# Is there is method to collect multiple types of elasticsearch logs when elasticsearchs and filebeats are deployed with ECK

**URL:** <https://discuss.elastic.co/t/is-there-is-method-to-collect-multiple-types-of-elasticsearch-logs-when-elasticsearchs-and-filebeats-are-deployed-with-eck/292656>\
**Category:** Beats\
**Tags:** docker, filebeat\
**Created:** [December 22, 2021, 9:01am UTC](https://discuss.elastic.co/t/is-there-is-method-to-collect-multiple-types-of-elasticsearch-logs-when-elasticsearchs-and-filebeats-are-deployed-with-eck/292656 "2021-12-22T09:01:02Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![bruce2](https://avatars.discourse-cdn.com/v4/letter/b/35a633/32.png) [@bruce2](https://discuss.elastic.co/u/bruce2)\
**Post date:** [December 22, 2021, 9:01am UTC](https://discuss.elastic.co/t/is-there-is-method-to-collect-multiple-types-of-elasticsearch-logs-when-elasticsearchs-and-filebeats-are-deployed-with-eck/292656/1 "2021-12-22T09:01:02Z")

</div>

I am using ECK1.6 to deploy Elasticsearch cluster 7.15.2 on k8s，and then using ECK to deploy filebeat to collect Elasticsearch server log, gc log, slowlog, deprecation log. There is an error as same as this issue [Filebeat registry collisions when an integration is used twice #22054](https://github.com/elastic/beats/issues/22054) when I using autodiscover feature and configure a template for echo types of log (server log, gc log, slowlog, deprecation log). While everything is OK when I comment out other three template and collect server logs only.  
So, Is there is method to collect multiple types of Elasticsearch logs when filebeat and Elasticsearch are deployed with ECK?  
My filebeat manifest is modified from [filebeat\_autodiscover\_by\_metadata.yaml](https://raw.githubusercontent.com/elastic/cloud-on-k8s/1.6/config/recipes/beats/filebeat_autodiscover_by_metadata.yaml)

---

<div class="post-metadata">

**Author:** ![mtojek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mtojek/32/63863_2.png) [@mtojek](https://discuss.elastic.co/u/mtojek)\
**Post date:** [December 23, 2021, 10:34am UTC](https://discuss.elastic.co/t/is-there-is-method-to-collect-multiple-types-of-elasticsearch-logs-when-elasticsearchs-and-filebeats-are-deployed-with-eck/292656/2 "2021-12-23T10:34:17Z")

</div>

Not sure if it helps, but worth trying to add IDs:

```auto
      - condition.equals.kubernetes.namespace: log-namespace
        config:
        - paths: ["/var/log/containers/*${data.kubernetes.container.id}.log"]
          type: container
          id: "first"
      - condition.equals.kubernetes.labels.log-label: "true"
        config:
        - paths: ["/var/log/containers/*${data.kubernetes.container.id}.log"]
          type: container
          id: "second"

```

---

<div class="post-metadata">

**Author:** ![bruce2](https://avatars.discourse-cdn.com/v4/letter/b/35a633/32.png) [@bruce2](https://discuss.elastic.co/u/bruce2)\
**Post date:** [December 24, 2021, 2:37am UTC](https://discuss.elastic.co/t/is-there-is-method-to-collect-multiple-types-of-elasticsearch-logs-when-elasticsearchs-and-filebeats-are-deployed-with-eck/292656/3 "2021-12-24T02:37:23Z")

</div>

Thank you.  
The same error is returned with "id" set. My filebeat.yaml is like this below:

```auto
      - condition
          equals:
            kubernetes:
              namespace: elastic-system
              labels:
                common_k8s_elastic_co/type: elasticsearch
        config:
        - module: elasticsearch
          slowlog:
            input:
              id: "slowlog"
              paths: ["/var/log/containers/*${data.kubernetes.container.id}.log"]
              type: container
              include_lines:
              - "type":\s"(index_search_slowlog|index_indexing_slowlog)"
      - condition
          equals:
            kubernetes:
              namespace: elastic-system
              labels:
                common_k8s_elastic_co/type: elasticsearch
        config:
        - module: elasticsearch
          server:
            input:
              id: "server"
              paths: ["/var/log/containers/*${data.kubernetes.container.id}.log"]
              type: container
              include_lines:
              - "type":\s"server"}

```

---

<div class="post-metadata">

**Author:** ![bruce2](https://avatars.discourse-cdn.com/v4/letter/b/35a633/32.png) [@bruce2](https://discuss.elastic.co/u/bruce2)\
**Post date:** [December 27, 2021, 8:13am UTC](https://discuss.elastic.co/t/is-there-is-method-to-collect-multiple-types-of-elasticsearch-logs-when-elasticsearchs-and-filebeats-are-deployed-with-eck/292656/4 "2021-12-27T08:13:56Z")

</div>

I find a perfect solution, it helped.

The solution is from this post:  
[https://xeraa.net/blog/2020\_filebeat-modules-with-docker-kubernetes/#planD](https://xeraa.net/blog/2020_filebeat-modules-with-docker-kubernetes/#planD)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 24, 2022, 10:13am UTC](https://discuss.elastic.co/t/is-there-is-method-to-collect-multiple-types-of-elasticsearch-logs-when-elasticsearchs-and-filebeats-are-deployed-with-eck/292656/5 "2022-01-24T10:13:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.