[SOLVED] Filebeat creates only one index


I'm using the default Filebeat configuration on a Kubernetes cluster. This is my configuration:

        - type: kubernetes
          host: ${NODE_NAME}
          hints.enabled: true
            type: container
              - /var/log/containers/*${data.kubernetes.container.id}.log

      - add_cloud_metadata:
      - add_host_metadata:
      - decode_json_fields:
          fields: ["message"]
          target: "json_message"
          process_array: true
          overwrite_keys: true

    cloud.id: ${ELASTIC_CLOUD_ID}
    cloud.auth: ${ELASTIC_CLOUD_AUTH}

      hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
      protocol: https
        certificate_authorities: /etc/ssl/certs/client/tls.crt

It works like a charm, the only issue is that Filebeat does not seem to create daily indices. It only created one, and it kept on using it on subsequent days. As far as I understand, Filebeat is supposed to create one index per day. This is the output of the _cat/shards request:

.kibana_task_manager_1           0 p STARTED        2  21.6kb elasticsearch-logging-es-default-2
.kibana_task_manager_1           0 r STARTED        2  21.6kb elasticsearch-logging-es-default-1
.kibana_1                        0 r STARTED        8 196.4kb elasticsearch-logging-es-default-2
.kibana_1                        0 p STARTED        8 196.4kb elasticsearch-logging-es-default-1
filebeat-7.4.2-2019.12.02-000001 0 r STARTED 32473955  18.6gb elasticsearch-logging-es-default-2
filebeat-7.4.2-2019.12.02-000001 0 p STARTED 32473434  18.8gb elasticsearch-logging-es-default-0
.security-7                      0 p STARTED       36  88.9kb elasticsearch-logging-es-default-0
.security-7                      0 r STARTED       36  88.9kb elasticsearch-logging-es-default-1
.apm-agent-configuration         0 p STARTED        0    283b elasticsearch-logging-es-default-2
.apm-agent-configuration         0 r STARTED        0    283b elasticsearch-logging-es-default-0

As you can see, the only index created is the one that was created when I deployed filebeat, 3 days ago.

I'm lost really. Should I add additional configuration for Filebeat to behave the way I expect it to behave?


Alright, this wasn't working because of ILM. I deactivated it and everything is working like a charm now.


This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.