I was able to make it work with a date in the index name, but by making the following modifications to my fluentd Kubernetes yaml file:
- name: FLUENT_ELASTICSEARCH_LOGSTASH_FORMAT
value: "false"
- name: FLUENT_ELASTICSEARCH_LOGSTASH_INDEX_NAME
value: "logstash-fluentd-kubernetes"
- name: FLUENT_ELASTICSEARCH_ENABLE_ILM
value: "true"
- name: FLUENT_ELASTICSEARCH_TEMPLATE_FILE
value: /host/index_template.json
- name: FLUENT_ELASTICSEARCH_TEMPLATE_NAME
value: "logstash-fluentd-kubernetes"
- name: FLUENT_ELASTICSEARCH_INCLUDE_TIMESTAMP
value: "true"
- name: FLUENT_ELASTICSEARCH_APPLICATION_NAME
value: "logging"
volumes:
- name: es-template
configMap:
name: es-template
volumeMounts:
- name: es-template
mountPath: /host
readOnly: true
---
apiVersion: v1
data:
index_template.json: |-
{
"index_patterns": [
"logstash-fluentd-kubernetes-*"
]
}
kind: ConfigMap
metadata:
name: es-template
namespace: kube-system
The index now look like this when I force a rollover:
logstash-fluentd-kubernetes-logging-2021.04.06-000002
logstash-fluentd-kubernetes-logging-2021.04.06-000001
... and it keeps writing to the same date it initially created the index with (not creating a new one everyday).
The index template name is: "logstash-fluentd-kubernetes
" and its pattern: "logstash-fluentd-kubernetes-*
"