FileBeat with CRI-O [not docker] container runtime environment on Kubernetes?

Hello,

I'm trying to deploy filebeat on Kubernetes that is configured with running CRI-O container runtime environment. Deploying the filebeat as in https://github.com/elastic/beats/blob/master/deploy/kubernetes/filebeat-kubernetes.yaml could not find the container log files.

So for cri-o based environment I have tried modifying the container log path for filebeat as in https://github.com/elastic/beats/issues/8985#issuecomment-437271804

apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-config
  namespace: kube-system
  labels:
    k8s-app: filebeat
data:
  filebeat.yml: |-
    filebeat.config:
      modules:
        path: ${path.config}/modules.d/*.yml
        # Reload module configs as they change:
        reload.enabled: false
    filebeat.autodiscover:
      providers:
        - type: kubernetes
          include_pod_uid: true
          in_cluster: true
          hints.enabled: true
          include_annotations: '*'
          templates:
            - condition.regexp:
                kubernetes.container.name: '.+'
              config:
                input:
                  type: container
                  json.keys_under_root: false
                  json.add_error_key: false
                  json.ignore_decoding_error: true
                  combine_partial: true
                 cri.parse_flags: true
                  cri.force: true
                  containers:
                    path: "/var/log/crio/pods/${data.kubernetes.pod.uid}/"
                    ids:
                      - "${data.kubernetes.container.name}"
    processors:
      - add_cloud_metadata: ~
      - add_kubernetes_metadata:
          in_cluster: true

    output.logstash:
      hosts: ['logstash:5044']

FileBeat Logs

2020-02-12T17:12:54.743Z INFO [autodiscover] autodiscover/autodiscover.go:105 Starting autodiscover manager 2020-02-12T17:12:54.845Z INFO log/input.go:152 Configured paths: [/var/lib/docker/containers/2c0aaeb30c01eb4cc7a187e4fa59f8777fce9b9947cad18c688cd5b43dcb655e/*-json.log] 2020-02-12T17:12:54.845Z INFO input/input.go:114 Starting input of type: container; ID: 15337647248318266633 2020-02-12T17:12:54.845Z INFO log/input.go:152 Configured paths: [/var/lib/docker/containers/ee3e3a0f173d0e39b741cc8b94b9f5e6843061418e7845b77981fe0a02bf6739/*-json.log] 2020-02-12T17:12:54.845Z INFO input/input.go:114 Starting input of type: container; ID: 11194825261634314163 2020-02-12T17:12:54.846Z INFO log/input.go:152 Configured paths: [/var/lib/docker/containers/a9cbc5469c305d5bcb2e5f6c1d50271314798075b9490fa1a1951a9d2b2a4ae3/*-json.log] 2020-02-12T17:12:54.846Z INFO input/input.go:114 Starting input of type: container; ID: 7915459815237518958 2020-02-12T17:12:54.846Z INFO log/input.go:152 Configured paths: [/var/lib/docker/containers/dfb185c4e6d39d5dd371f27422f97d118bf9d307e6e69a49c0a3cb3be0ef30b3/*-json.log] 2020-02-12T17:12:54.846Z INFO input/input.go:114 Starting input of type: container; ID: 14976356031607412965 2020-02-12T17:12:54.847Z INFO log/input.go:152 Configured paths: [/var/lib/docker/containers/60f89f881ec85239470d900d7e5e53b3ad4fcf59ec46879e6c9c656a5bf4eedf/*-json.log] 2020-02-12T17:12:54.847Z INFO input/input.go:114 Starting input of type: container; ID: 9390979353581095752 2020-02-12T17:12:54.847Z INFO log/input.go:152 Configured paths: [/var/lib/docker/containers/7ff92e3a3b7d8b7082db39bc68f13b8cf01c3f7ee8aa72eb7fc584bd92b27886/*-json.log] 2020-02-12T17:12:54.847Z INFO input/input.go:114 Starting input of type: container; ID: 10588676206953125327 2020-02-12T17:12:54.848Z INFO log/input.go:152 Configured paths: [/var/lib/docker/containers/2d4eda33c94114f0654989dcd64643dcfc4e9929205de00eebe4a45ba20c06e1/*-json.log] 2020-02-12T17:12:54.848Z INFO input/input.go:114 Starting input of type: container; ID: 12414274566107568665 2020-02-12T17:12:54.848Z INFO log/input.go:152 Configured paths: [/var/lib/docker/containers/19255798918a11a12375f6c5b677fccaf13afdb026846d02c6542dd11aa2a0bb/*-json.log] 2020-02-12T17:12:54.848Z INFO input/input.go:114 Starting input of type: container; ID: 17154145792288559585 2020-02-12T17:12:54.849Z INFO log/input.go:152 Configured paths: [/var/lib/docker/containers/262456cd7d6d9d2e1712090a37817d34f8f6dd9289f675f66f80f875c5689fec/*-json.log] 2020-02-12T17:12:54.849Z INFO input/input.go:114 Starting input of type: container; ID: 11997798302486402131 2020-02-12T17:12:54.849Z INFO log/input.go:152 Configured paths: [/var/lib/docker/containers/fd9c3ef2cb858b07f79d48508bc2899ebcbab1674466f1efbcd9be1385ee252a/*-json.log] 2020-02-12T17:12:54.849Z INFO input/input.go:114 Starting input of type: container; ID: 9689899642930122113 2020-02-12T17:12:54.850Z INFO log/input.go:152 Configured paths: [/var/lib/docker/containers/c23d93f667f2476c10dfd34409e9ffc68eac0de91a64d6fde359384ac3be7b85/*-json.log] 2020-02-12T17:12:54.850Z INFO input/input.go:114 Starting input of type: container; ID: 8237671109160429285 2020-02-12T17:12:54.850Z INFO log/input.go:152 Configured paths: [/var/lib/docker/containers/d507f69c1a147410885f94c7a40297386426448037c50780ddad8d5eff51e8aa/*-json.log] 2020-02-12T17:12:54.850Z INFO input/input.go:114 Starting input of type: container; ID: 5030918935259973164 2020-02-12T17:12:54.851Z INFO log/input.go:152 Configured paths: [/var/lib/docker/containers/72361a57dda867b4819b08394d2d9c3116a64fb2085a691123dc863f84205f74/*-json.log] 2020-02-12T17:12:54.851Z INFO input/input.go:114 Starting input of type: container; ID: 15961752709069961090 2020-02-12T17:12:54.851Z INFO log/input.go:152 Configured paths: [/var/lib/docker/containers/ffa1c77cace5414c84175573acb77e7ee6a1ae2e6d6f78941fd3bf5e4c1229d8/*-json.log] 2020-02-12T17:12:54.851Z INFO input/input.go:114 Starting input of type: container; ID: 3494905796133945119 2020-02-12T17:12:54.852Z INFO log/input.go:152 Configured paths: [/var/lib/docker/containers/79143c8c234b985a1e98c32954ab5c0344779005901ca59f5758a99f55cfae28/*-json.log] 2020-02-12T17:12:54.852Z INFO input/input.go:114 Starting input of type: container; ID: 17402523985284215498
`
but it's still looking at docker path [/var/lib/docker/] and it's neither harvesting the log files nor forwarding them to logstash ?

Appreciate any help

Thanks.

Got resolved from



This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.