EFK Stack on Kubernetes - Collecting logs from default namespace

Hi, I am using EFK stack on Kubernetes, I want to configure fluentd to collect logs from one specific namespace, the default namespace.
This is my fleuntd config file:

<label @FLUENT_LOG>
      <match fluent.**>
        @type null
      </match>
    </label>

    <source>
      @type tail
      path /var/log/containers/*.log
      pos_file /var/log/app.log.pos
      tag kubernetes.*
      read_from_head true
      <parse>
        @type json
        time_format %Y-%m-%dT%H:%M:%S.%NZ
      </parse>
    </source>

    <filter kubernetes.**>
        @type kubernetes_metadata
    </filter>

    <filter kubernetes.**>
       @type grep
         <exclude>
            key log
            pattern (.\[notice]\.*|^[ \\\/\(\)\*\|_]+(?!.*[a-zA-Z0-9]).*$|^\s*$|.*GET*|.*POST*)
         </exclude>
         <regexp>
            key kubernetes.namespace_name
            pattern /^default$/
         </regexp>
    </filter>
    <match ** >
       @type elasticsearch_dynamic
       @log_level info
       include_tag_key true
       host "#{ENV['FLUENT_ELASTICSEARCH_HOST']}"
       port "#{ENV['FLUENT_ELASTICSEARCH_PORT']}"
       user "#{ENV['FLUENT_ELASTICSEARCH_USER']}"
       password "#{ENV['FLUENT_ELASTICSEARCH_PASSWORD']}"
       scheme "#{ENV['FLUENT_ELASTICSEARCH_SCHEME'] || 'http'}"
       ssl_verify "#{ENV['FLUENT_ELASTICSEARCH_SSL_VERIFY'] || 'true'}"
       reload_connections true
       logstash_format true
       logstash_prefix logstash
       <buffer>
           @type file
           path /var/log/fluentd-buffers/kubernetes.system.buffer
           flush_mode interval
           retry_type exponential_backoff
           flush_thread_count 2
           flush_interval 5s
           retry_forever true
           retry_max_interval 30
           chunk_limit_size 2M
           queue_limit_length 32
           overflow_action block
       </buffer>
    </match>

The way I tired to do that is by adding:

         <regexp>
            key kubernetes.namespace_name
            pattern /^default$/
         </regexp>

but is not working this way, fluentd is not collecting any logs.

I don't know if the question is for here but please help me if you know the answer.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.