Filebeat 7.3.0 High CPU

Elastic Stack Beats
1

Hi,

I am not sure if this is the best place to discuss this but one of our compute nodes in Kubernetes is constantly alerting on high CPU. After digging a bit deeper, we see that the Filebeat process on the node is triggering the alert. Here are some configuration and metrics:

Filebeat version: 7.0.1
Filebeat configuration with sensitive information redacted:

image:
  repository: docker.elastic.co/beats/filebeat-oss
  tag: 7.3.0
  pullPolicy: IfNotPresent

config:
  logging.json: true
  filebeat.config:
    filebeat.autodiscover:
      providers:
       - type: kubernetes
         hints.enabled: true

  filebeat.inputs:
    - type: log
      enabled: true
      paths:
        - /var/log/*.log
        - /var/log/messages
        - /var/log/syslog
    - type: docker
      json.keys_under_root: false
      json.add_error_key: true
      containers.ids:
        - "*"
      processors:
        - add_kubernetes_metadata:
            in_cluster: true
        - drop_event:
            when:
              equals:
                kubernetes.container.name: "filebeat"

  # Disable automatic ILM, since we already manage it separately
  setup:
    ilm:
      enabled: false

    template:
      name: "<template_name>"
      pattern: "<pattern>"

  output:
    elasticsearch:
      hosts: ["<elasticsearch_endpoint>"]
      username: "<username>"
      # password: "set me through --set config.output.elasticsearch.password=XYZ"
      enable: true
      bulk_max_size: 1000
      index: "<index>"
      indices:
        - index: "<index>"
      path: "/"

  output.file:
      enabled: false

  http.enabled: false

monitoring:
  enabled: false

We've not made any changes to Filebeat itself recently. Additionally, the Filebeat logs do not hint at the underlying issue. What else can we look at to figure out a root case for the high CPU alert/s? As an additional debugging step, I did look at the following documents but am not sure if they are relevant:

I am happy to provide additional details as needed so any help would be greatly appreciated.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.