Configure filebeat to ship elasticsearch logs to monitoring cluster in kubernetes

Elastic Stack Beats
,
1

I've 2 clusters (7.10) running on Kubernetes - 1 prod and 1 monitoring. I've successfully set up metricbeat to ship elasticsearch metrics to my monitoring cluster. Now I want to ship elasticsearch logs using filebeat to my monitoring cluster. I have tried this filebeat config:

filebeat.modules:
  - module: elasticsearch
    server:
      enabled: true
filebeat.autodiscover:
          providers:
          - type: kubernetes
            node: ${NODE_NAME}
            templates:
              - condition:
                  contains:
                    kubernetes.container.name: "elasticsearch"
                  host: ${HOSTNAME}
                config:
                  - type: container
                    paths:
                      - "/var/log/containers/*-${data.kubernetes.container.id}.log"
                    json.keys_under_root: true
                    json.add_error_key: true
                    json.message_key: message
        
        output.elasticsearch:
          hosts: ["http://es-monitoring-master.es-monitoring.svc.cluster.local:9200"]

but the Stack Monitoring page on my monitoring cluster still says "No Logs Found" although I can see filebeat index created in my monitoring cluster. What am I missing?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.