Log messages not populated in Elasticsearch

Elastic Stack Beats
1

Hi, following is my filebeat config and I am trying to push Kubernetes container logs to Elastic search, unfortunately log messages are not loaded . Could someone please help me what I am missing in my config?

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    beat.k8s.elastic.co/name: quickstart
    common.k8s.elastic.co/type: beat
    eck.k8s.elastic.co/credentials: "true"
    k8s-app: filebeat
  name: quickstart-beat-filebeat-config
  namespace: default
data:
  beat.yml: |-
    filebeat.inputs:
    - type: filestream
      id: kubernetes-container-logs
      paths:
        - /var/log/containers/quicksilver-*.log
      parsers:
        - container: ~
      prospector:
        scanner:
          fingerprint.enabled: true
          symlinks: true
      file_identity.fingerprint: ~    
      processors:
        - add_kubernetes_metadata:  
            in_cluster: true 
            default_matchers.enabled: true
    processors:
      - add_cloud_metadata:
      - add_host_metadata:
    logging:
        files:
            keepfiles: 7
            name: filebeat.log
            path: /var/log/filebeat
            permissions: 420
        level: debug
        to_files: true
    output:
        elasticsearch:
            hosts: [http://elastic-quickcore-ws-http.default.svc:9200]
            username: default-quickstart-beat-user
            password: pdS4reIh292U3m9B9U0MJV36
            ssl:
                certificate_authorities:
                    - /mnt/elastic-internal/elasticsearch-certs/ca.crt

Screenshot 2025-02-26 at 6.19.15 PM
Screenshot 2025-02-26 at 6.19.15 PM2556×636 55.4 KB

2

Can you share a sample log lines grabbed directly from the container?

And also the full document for that log line in Kibana? The screenshot has the headers cut off and doesn't contain much information. Another option might be to just query the _search endpoint for the filebeat-* datastream via Dev Tools by running POST filebeat-*/_search and sharing the results so we can see what fields are present in the document.

3

Hello,

Could you please check the filebeat logs and the server which can help if there is any error while processing the records. If there are no errors in the log file then we need to review the YML file which could be an issue.

Thanks!!

4

Issue is resolved