Kubernetes Filebeat (7.4.2) Chart (7.4.1) + autodiscover results in Error creating runner from config

[Alexei_Smirnov]

I've been trying to setup FILEBEAT in our kubernetes clusters. Tried using filebeat chart from elastic helm repo, it is currently at 7.4.1 and also tried to upgrade to image 7.4.2 to same result.

We are using filebeat.autodiscover with provider: kubernetes . tried default_config and tried adding templates for NGINX / Redis, but keep on getting the following error:

[beats-filebeat-xd5nt] 2019-11-23T12:48:09.623Z ERROR [autodiscover] cfgfile/list.go:96 Error creating runner from config: Can only start an input when all related states are finished: {Id:1574005-2049 Finished:false Fileinfo:0xc0006384e0 Source:/var/log/containers/beats-filebeat-xd5nt_kube-system_filebeat-09295df339975c5ce49c850f2104e8d0d57967edae58df175a5d84e0be42d31f.log Offset:598 Timestamp:2019-11-23 12:47:55.930959861 +0000 UTC m=+1.606712406 TTL:-1ns Type:container Meta:map[] FileStateOS:1574005-2049}

here's a snippet from the filebeat.yml config:

logging.level: warning

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

filebeat.autodiscover:
  providers:
    - type: kubernetes
      host: ${NODE_NAME}
      hints.enabled: true
      hints.default_config:
        type: container
        paths:
          - /var/log/containers/*${data.kubernetes.container.id}.log


processors:
  - drop_event:
      when.or:
          - and:
              - regexp:
                  message: '^\d+\.\d+\.\d+\.\d+ '
              - equals:
                  fileset.name: error
          - and:
              - not:
                  regexp:
                      message: '^\d+\.\d+\.\d+\.\d+ '
              - equals:
                  fileset.name: access

  - add_fields:
      target: 'project'
      fields:
        env: '${ENVIRONMENT}'
        name: '${PROJECT}'

  - add_tags:
      tags: ['${PROJECT}', '${ENVIRONMENT}']
      target: "environment"

  - rename:
      fields:
        - {from: "message", to: "event_original"}

  - decode_cef:
      field: event_original
      ignore_missing: true
      ignore_failure: true
      target_field: cef

  - decode_json_fields:
      when:
        regexp:
          event_original: '^{.*}$'
      fields: ["event_original"]
      process_array: true
      max_depth: 10
      target: "json"
      overwrite_keys: true
      add_error_key: true

  - rename:
      fields:
        - {from: "json.method", to: "method"}
        - {from: "json.duration", to: "duration"}
        - {from: "json.message", to: "message"}
      ignore_missing: true
      fail_on_error: false

  - drop_fields:
      fields: ['json.responseHeaders', 'json.requestHeaders', 'agent']

  - add_kubernetes_metadata:
      in_cluster: true