Filebeat.autodiscover

[sszabo (Steve Szabo)]

If we have a single filebeat pod running on a kubernetes worker node with two inputs both using the same container id list will this cause a problem?

    filebeat.autodiscover:
    providers:
    - type: kubernetes
      templates:

        - config:
          - type: docker
            cri:
              parse_flags: true
              force: true
            containers:
              path: /var/data/kubeletlogs
              ids:
                - "/${data.kubernetes.pod.uid}/${data.kubernetes.container.name}"
            fields_under_root: true
            fields:
              oiqformat: infra
          condition:
            not:
              equals:
                kubernetes.labels.oiqlogformat: ddiq

        - config:
          - type: docker
            cri:
              parse_flags: true
              force: true
            containers:
              path: /var/data/kubeletlogs
              ids:
                - "/${data.kubernetes.pod.uid}/${data.kubernetes.container.name}"
            fields_under_root: true
            fields:
              oiqformat: ddiq
            multiline.pattern: '^(INFO|WARN|FATAL|ERR|DEBUG|{)'
            multiline.negate: true
            multiline.match: after
          condition:
            equals:
              kubernetes.labels.oiqlogformat: ddiq

[sszabo (Steve Szabo)]

We get this in the filebeat log:

2019-06-25T18:20:50.014Z INFO log/input.go:148 Configured paths: [/var/data/kubeletlogs/f4bf8da5-9775-11e9-8dcd-dac7c637fa8b/filebeat/*.log]
2019-06-25T18:20:50.014Z INFO input/input.go:114 Starting input of type: docker; ID: 3753679483445256937
2019-06-25T18:20:50.015Z INFO log/harvester.go:254 Harvester started for file: /var/data/kubeletlogs/f4bf8da5-9775-11e9-8dcd-dac7c637fa8b/filebeat/0.log

..
..

2019-06-25T18:30:50.016Z INFO kubernetes/watcher.go:263 kubernetes: Unexpected EOF while watching API
2019-06-25T18:30:50.016Z INFO kubernetes/watcher.go:242 kubernetes: Watching API for resource events
2019-06-25T18:30:50.113Z ERROR [autodiscover] cfgfile/list.go:96 Error creating runner from config: Can only start an input when all related states are finished: {Id:921155-64512 Finished:false Fileinfo:0xc0001021a0 Source:/var/data/kubeletlogs/f4bf8da5-9775-11e9-8dcd-dac7c637fa8b/filebeat/0.log Offset:32903 Timestamp:2019-06-25 18:30:30.620916281 +0000 UTC m=+580.955716168 TTL:-1ns Type:docker Meta:map FileStateOS:921155-64512}
2019-06-25T18:30:50.113Z INFO input/input.go:149 input ticker stopped
2019-06-25T18:30:50.113Z INFO input/input.go:167 Stopping Input: 3753679483445256937
2019-06-25T18:30:50.113Z INFO log/harvester.go:275 Reader was closed: /var/data/kubeletlogs/f4bf8da5-9775-11e9-8dcd-dac7c637fa8b/filebeat/0.log. Closing.