Can't get Filebeat to ship Nginx Ingress Controller logs using ECK

Elastic Stack Beats
,
1

I have ECK setup and running on my kubernetes cluster. I followed the Configuration Examples to setup filebeat ship all container logs to Elasticsearch. That is working fine.

However, now I am trying to parse the logs of my Nginx Ingress Controller using the nginx filebeat module so that I can better search the ingress logs. But I just can't seem to get it to work, no matter how many different examples I see online. This is what my Beat resource looks like:

apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
  name: container-logs
  namespace: elasticsearch
spec:
  config:
    filebeat:
      autodiscover:
        providers:
          - hints:
              default_config:
                enabled: false
            node: ${NODE_NAME}
            templates:
              - config:
                  - paths:
                      - /var/log/containers/*${data.kubernetes.container.id}.log
                    type: container
              - condition:
                  equals:
                    kuberentes.container.name: controller
                config:
                  - ingress_controller:
                      enabled: true
                      vars:
                        paths:
                          - /var/log/containers/*${data.kubernetes.container.id}.log
                    module: nginx
            type: kubernetes
    processors:
      - add_cloud_metadata: {}
      - add_host_metadata: {}
    setup:
      dashboards:
        enabled: false
  daemonSet:
    podTemplate:
      spec:
        automountServiceAccountToken: true
        containers:
          - env:
              - name: NODE_NAME
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
            name: filebeat
            volumeMounts:
              - mountPath: /var/log/containers
                name: varlogcontainers
              - mountPath: /var/log/pods
                name: varlogpods
              - mountPath: /var/lib/docker/containers
                name: varlibdockercontainers
        dnsPolicy: ClusterFirstWithHostNet
        hostNetwork: true
        securityContext:
          runAsUser: 0
        serviceAccountName: filebeat
        volumes:
          - hostPath:
              path: /var/log/containers
            name: varlogcontainers
          - hostPath:
              path: /var/log/pods
            name: varlogpods
          - hostPath:
              path: /var/lib/docker/containers
            name: varlibdockercontainers
  elasticsearchRef:
    name: elasticsearch
  kibanaRef:
    name: kibana
  monitoring:
    logs: {}
    metrics:
      elasticsearchRefs:
        - name: elasticsearch
          namespace: elasticsearch
  type: filebeat
  version: 8.10.1

Has anyone else gotten this to work on their setup? I feel like I've tried everything.

2

Hi @krische

Hmm just a quick look are you missing the provider type

  config:
    filebeat.autodiscover.providers:
    - node: ${NODE_NAME}
      type: kubernetes <!--- Looks like you are missing this? 
      hints.default_config.enabled: "false"
      templates:
      - condition.equals.kubernetes.namespace: log-namespace
        config:
        - paths: ["/var/log/containers/*${data.kubernetes.container.id}.log"]
          type: container
      - condition.equals.kubernetes.labels.log-label: "true"
        config:
        - paths: ["/var/log/c

Also you can read more about filebeat autodiscover here...

3

It looks like part of my issue was a typo in the condition, I had "kuberentes" instead of "kubernetes".

And I don't know if it was necessary, but I also overrode some of the input stuff. So this is what ended up working for me:

apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
  name: container-logs
  namespace: elasticsearch
spec:
  config:
    filebeat:
      autodiscover:
        providers:
          - hints:
              default_config:
                enabled: false
            node: ${NODE_NAME}
            templates:
              - config:
                  - paths:
                      - /var/log/containers/*${data.kubernetes.container.id}.log
                    type: container
              - condition:
                  equals:
                    kubernetes.container.name: controller
                config:
                  - ingress_controller:
                      enabled: true
                      input:
                        paths:
                          - /var/log/containers/*${data.kubernetes.container.id}.log
                        type: container
                      vars:
                        paths:
                          - /var/log/containers/*${data.kubernetes.container.id}.log
                    module: nginx
            type: kubernetes
    processors:
      - add_cloud_metadata: {}
      - add_host_metadata: {}
    setup:
      dashboards:
        enabled: false
  daemonSet:
    podTemplate:
      spec:
        automountServiceAccountToken: true
        containers:
          - env:
              - name: NODE_NAME
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
            name: filebeat
            volumeMounts:
              - mountPath: /var/log/containers
                name: varlogcontainers
              - mountPath: /var/log/pods
                name: varlogpods
              - mountPath: /var/lib/docker/containers
                name: varlibdockercontainers
        dnsPolicy: ClusterFirstWithHostNet
        hostNetwork: true
        securityContext:
          runAsUser: 0
        serviceAccountName: filebeat
        volumes:
          - hostPath:
              path: /var/log/containers
            name: varlogcontainers
          - hostPath:
              path: /var/log/pods
            name: varlogpods
          - hostPath:
              path: /var/lib/docker/containers
            name: varlibdockercontainers
  elasticsearchRef:
    name: elasticsearch
  kibanaRef:
    name: kibana
  monitoring:
    logs: {}
    metrics:
      elasticsearchRefs:
        - name: elasticsearch
          namespace: elasticsearch
  type: filebeat
  version: 8.10.1
1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.