Kubernetes autodiscover indices

Elastic Stack Beats
1

Hi

I am trying to setup filebeat with kubernetes so all the logs of the pods in all the different namespaces get sent to elasticsearch. But I would like different indexes for every app. That is why I use the %{[kubernetes.labels.app]} in the elastic output configuration. This works, but when i check the index management all new indexes have lifecycle errors.
Is there a way to automatically create lifecycles for every app in this way of is a different approach needed

filebeat.autodiscover:
  providers:
    - type: kubernetes
      host: ${NODE_NAME}
      hints.enabled: true
      hints.default_config:
        type: container
        paths:
          - /var/log/containers/*${data.kubernetes.container.id}.log

processors:
  - add_cloud_metadata:
  - add_host_metadata:
  - add_kubernetes_metadata:

cloud.id: ${ELASTIC_CLOUD_ID}
cloud.auth: ${ELASTIC_CLOUD_AUTH}

output.elasticsearch:
  hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
  username: ${ELASTICSEARCH_USERNAME}
  password: ${ELASTICSEARCH_PASSWORD}
  indices:
    - index: "filebeat-%{[agent.version]}-%{[kubernetes.labels.app]}-%{+yyyy-MM-dd}"
      when.and:
        - not.equals:
            kubernetes.namespace: "kube-system"
        - not.equals:
            kubernetes.namespace: "default"
        - not.equals:
            kubernetes.namespace: "cert-manager"
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.