Filebeat harvester unable to open file and unable to send logs to logstash

Hello Folks,
I have installed Elastic filebeat as a sidecar container in kubernetes cluster with main application "Sonarqube". I want to ships the sonarqube logs to logstash. Filebeat container is running and reading some metrics but not sending logs.

Your support really help me get through to this.

// values.yaml

  edition: enterprise

  postgresql:
    enabled: false

  jdbcOverwrite:
    enable: true
    jdbcUsername: "username"
    jdbcSecretName: "secretname"
    jdbcSecretPasswordKey: "password"
    jdbcUrl: "jdbc:postgresql://servername:5432"

  ingress:
    enabled: true
    ingressClassName: nginx
    tls:
    - secretName: soanrqube-certificate
      hosts:
      - sonarqube-dev.xyz.com
    hosts:
    - name: sonarqube-dev.xyz.com
  extraContainers:
    - name: filebeat-sonar
      image: docker.elastic.co/beats/filebeat:8.3.3
      imagePullPolicy: Always
      volumeMounts:
        - name: sonarqube
          mountPath: /opt/sonarqube/logs
          subPath: logs
        - name: filebeat-config
          mountPath: /etc/filebeat.yml
          subPath: filebeat.yml
          readOnly: true
  persistence:
    volumes:
      - name: sonarqube
        emptyDir: {}
      - name: filebeat-config
        configMap:
          name: filebeat-configmap
          defaultMode: 0600

// filebeat-configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-configmap
data:
    filebeat.yml: |-
      filebeat.inputs:
      - type: container
        enabled: true
        paths:
          - /opt/sonarqube/logs/*
      output.logstash:
        hosts: ["logstashhost:5044"]

// filebeat pod's logs

{"log.level":"info","@timestamp":"2022-09-05T08:21:54.173Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":185},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"cpuacct":{"total":{"ns":10679378}},"memory":{"mem":{"usage":{"bytes":-118784}}}},"cpu":{"system":{"ticks":30},"total":{"ticks":150,"time":{"ms":10},"value":0},"user":{"ticks":120,"time":{"ms":10}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":9},"info":{"ephemeral_id":"57e6a626-6bc3-4086-ad29-3fba60617892","uptime":{"ms":270068},"version":"8.3.3"},"memstats":{"gc_next":18201160,"memory_alloc":8909856,"memory_total":53916472,"rss":103178240},"runtime":{"goroutines":27}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":0,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.42,"15":0.15,"5":0.24,"norm":{"1":0.0525,"15":0.0188,"5":0.03}}}},"ecs.version":"1.6.0"}}

You may want to enable debug logs to see more details about what's under the hood.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.