Filebeat - How to disable "kube-system" namespace logs?

Elastic Stack Beats
1

Hello, I am using Elasticsearch + Filebeat and Kibana. I have two namespaces "datastore" and "logging". I would like only to collect logs from those and avoid to harvest log from "kube-system".

Checking in kibana I am able to see a lot of messages about kubernetes cluster being collected. (image attached).

k8s_namespace|407x500

Thanks in advance!

I tried multiples configurations. For example:

     filebeat.yml: |
  filebeat.autodiscover:
     providers:
      - type: kubernetes
        node: ${NODE_NAME}
        hints.enabled: true
        templates:
          - condition:
              or:
                - equals:
                    kubernetes.namespace: datastore
                - equals:
                    kubernetes.namespace: logging

  output.elasticsearch:
    host: '${NODE_NAME}'
    hosts: '${ELASTICSEARCH_HOSTS:elasticsearch-master:9200}'
2

Configuration looks ok. Can you try removing the or clause and checking with a single namespace? Maybe the or clause isn't working as expected.

3

Hello @Mario_Castro,

I tried some variations here. I found a incorrect syntax reading documentation. I was using "- equals:" now I replaced to "equals:" without "-"

I tried this way:

filebeatConfig:
  filebeat.yml: |
     filebeat.autodiscover:
       providers:
         - type: kubernetes
           templates:
             - condition:
                 equals:
                   kubernetes.namespace: datastore
                 equals:
                   kubernetes.namespace: logging
               config:
                 - type: log
                   paths:
                     - /opt/data/indexing-logs/*.log
     output.elasticsearch:
       host: '${NODE_NAME}'
       hosts: '${ELASTICSEARCH_HOSTS:elasticsearch-master:9200}'

After trigger a task in apache druid I can to observe "Harvester started" but nothing related to "/opt/data/indexing-logs/*.log":

filebeat-filebeat-f7vg6 filebeat 2020-12-22T20:07:24.706Z	INFO	[monitoring]	log/log.go:145	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":4450,"time":{"ms":249}},"total":{"ticks":15470,"time":{"ms":868},"value":15470},"user":{"ticks":11020,"time":{"ms":619}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":20},"info":{"ephemeral_id":"b7efd4ee-13ed-42f2-b97b-a70f72415f49","uptime":{"ms":570097}},"memstats":{"gc_next":29568544,"memory_alloc":18080344,"memory_total":1230986328,"rss":-2179072},"runtime":{"goroutines":299}},"filebeat":{"events":{"active":-1,"added":4724,"done":4725},"harvester":{"files":{"05788397-ef41-4997-a4d8-1f333d59c409":{"last_event_published_time":"2020-12-22T20:07:15.253Z","last_event_timestamp":"2020-12-22T20:07:06.005Z","name":"/var/lib/docker/containers/cb4fb822c0f3c06f5108313edd1b0326d8cf8e6188addf4a01f8b874a311ac62/cb4fb822c0f3c06f5108313edd1b0326d8cf8e6188addf4a01f8b874a311ac62-json.log","read_offset":57137,"size":57137,"start_time":"2020-12-22T20:07:15.250Z"},"09d094b9-00dc-4a57-b701-79dedf7b5517":{"last_event_published_time":"2020-12-22T20:07:23.307Z","last_event_timestamp":"2020-12-22T20:07:17.786Z","read_offset":591,"size":394},"135b0a13-7217-4958-9b4f-d1fbd67b9209":{"last_event_published_time":"2020-12-22T20:07:20.559Z","last_event_timestamp":"2020-12-22T20:07:20.190Z","read_offset":69143,"size":67624},"b40839d2-ff3f-4603-a858-29e2d117ff22":{"last_event_published_time":"2020-12-22T20:07:12.331Z","last_event_timestamp":"2020-12-22T20:07:12.037Z","read_offset":310,"size":465},"deb55111-0b43-4c0c-abc3-77aa3fb1f65d":{"last_event_published_time":"2020-12-22T20:07:13.303Z","last_event_timestamp":"2020-12-22T20:07:11.844Z","read_offset":394,"size":393},"e0c2c907-90cd-4acf-96c0-e035e22ee295":{"last_event_published_time":"2020-12-22T20:07:13.295Z","last_event_timestamp":"2020-12-22T20:07:12.934Z","read_offset":394,"size":394},"ee319a57-c8e0-454e-9aa6-3643a7ffcd0c":{"last_event_published_time":"2020-12-22T20:07:05.257Z","last_event_timestamp":"2020-12-22T20:07:05.009Z","read_offset":569,"size":569},"f06ce979-6661-4344-89f9-5768a1856484":{"last_event_published_time":"2020-12-22T20:07:21.276Z","last_event_timestamp":"2020-12-22T20:07:20.502Z","read_offset":1022556,"size":1023046}},"open_files":8,"running":8,"started":1}},"libbeat":{"config":{"module":{"running":17}},"output":{"events":{"acked":4724,"batches":101,"total":4724},"read":{"bytes":63721},"write":{"bytes":6467002}},"pipeline":{"clients":17,"events":{"active":0,"filtered":1,"published":4723,"total":4724},"queue":{"acked":4724}}},"registrar":{"states":{"current":16,"update":4725},"writes":{"success":112,"total":112}},"system":{"load":{"1":1.35,"15":1.54,"5":1.68,"norm":{"1":0.1688,"15":0.1925,"5":0.21}}}}}}
4

Oh yeah, you don't need -. Can you try with a single equals clause, just to see if it works with the rest of the setup and gives the expected output

5

Hello @Mario_Castro I tried yesterday.

I am planning to test fluentd because unfortunately I am not able to make filebeat to work as expected.

If you have any idea please advise.

Thanks.

6

I don't understand very well, your first example don't have a config path, but your second has it. It's important to provide the full yaml to see potential mistakes. Maybe you can try something like this? Or you already tried it?

filebeat.autodiscover:
   providers:
    - type: kubernetes
      node: ${NODE_NAME}
      hints.enabled: true
      templates:
        - condition:
             or:
                - equals:
                    kubernetes.namespace: datastore
                - equals:
                    kubernetes.namespace: logging
          config:
              - type: log
                paths:
                   - /opt/data/indexing-logs/*.log
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.