Error: setting 'filebeat.config.prospectors' has been removed after upgrade to 7.2

[aleksei.saiko]

Hi there!
I started to upgrade my ELK stack form 6.6.2 to 7.2.0.

Changed accordingly Elastic and Kibana configs , but got stuck on Filebeat.

After deploy to K8S, I receive this error

error: setting 'filebeat.config.prospectors' has been removed

My Conf file -

apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-config
  namespace: kube-system
  labels:
    k8s-app: filebeat
    kubernetes.io/cluster-service: "true"
data:
  filebeat.yml: |-
    filebeat.config:
      prospectors:
        # Mounted `filebeat-prospectors` configmap:
        path: ${path.config}/prospectors.d/*.yml
        # Reload prospectors configs as they change:
        reload.enabled: false
      modules:
        path: ${path.config}/modules.d/*.yml
        # Reload module configs as they change:
        reload.enabled: false

    output.logstash:
      hosts: ${LOGSTASH_HOSTS:?No logstash host configured. Use env var LOGSTASH_HOSTS to set hosts.}

My Prodpectors -

apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-prospectors
  namespace: kube-system
  labels:
    k8s-app: filebeat
    kubernetes.io/cluster-service: "true"
data:
  kubernetes.yml: |-
    - type: container
      paths:
        - /var/lib/docker/containers/*/*.log
      processors:
        - add_kubernetes_metadata:
            in_cluster: true
      scan_frequency: 10s
      close_inactive: 1m

After the upgrade, in Prospectors, I changed the -

- type: docker
      containers.ids:
      - "*"

to

- type: container
      paths:
        - /var/lib/docker/containers/*/*.log

My DemonSet file -

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: filebeat
  namespace: kube-system
  labels:
    k8s-app: filebeat
    kubernetes.io/cluster-service: "true"
spec:
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        k8s-app: filebeat
        kubernetes.io/cluster-service: "true"
    spec:
      serviceAccountName: filebeat
      terminationGracePeriodSeconds: 30
      containers:
      - name: filebeat
        image: docker.elastic.co/beats/filebeat:7.2.0
        args: [
          "-c", "/etc/filebeat.yml",
          "-e",
        ]
        env:
        - name: LOGSTASH_HOSTS
          value: elastic.XXXX.pro:5046
        securityContext:
          runAsUser: 0
        resources:
          limits:
            memory: 200Mi
          requests:
            cpu: 100m
            memory: 100Mi
        volumeMounts:
        - name: config
          mountPath: /etc/filebeat.yml
          readOnly: true
          subPath: filebeat.yml
        - name: prospectors
          mountPath: /usr/share/filebeat/prospectors.d
          readOnly: true
        - name: data
          mountPath: /usr/share/filebeat/data
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
        - name: varlog
          mountPath: /var/log
          readOnly: true
      volumes:
      - name: config
        configMap:
          defaultMode: 0600
          name: filebeat-config
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers
      - name: prospectors
        configMap:
          defaultMode: 0600
          name: filebeat-prospectors
      - name: data
        emptyDir: {}

Please advice, I don't see what else to change to get rid of this error.

Thanks!

[faec (Fae Charlton)]

Hi! Sorry for the confusion, in version 7 prospectors has been renamed to inputs. There are a few other changes, you seem to have already found some of them, but generally whatever used to go under filebeat.prospectors should go under filebeat.inputs.

[aleksei.saiko]

@faec

ilebe

Yeah, I figured it out, replaced to filbeat.inputs and deployed successfully.