# Filebeat not creating index daily **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [July 30, 2020, 8:21am UTC](https://discuss.elastic.co/t/filebeat-not-creating-index-daily/243181 "2020-07-30T08:21:01Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![daniel\_fablius](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_fablius/32/67143_2.png) [@daniel\_fablius](https://discuss.elastic.co/u/daniel_fablius)\ **Post date:** [July 30, 2020, 8:21am UTC](https://discuss.elastic.co/t/filebeat-not-creating-index-daily/243181/1 "2020-07-30T08:21:01Z")
Hi, so i use filebeat to send logs to Elasticsearch and i see an index is created with name "filebeat-7.7.1-2020.07.29-000002" i assume that filebeat will keep creating a new index based on the datetime of the data event is sent. but on 30th July 2020, no new index is created. is what i assume is wrong? is filebeat not supposed to create a new index everyday? if what i assume is right, what i should do to make it happened? ---
**Author:** ![fgjensen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fgjensen/32/62320_2.png) [@fgjensen](https://discuss.elastic.co/u/fgjensen)\ **Post date:** [July 30, 2020, 8:26am UTC](https://discuss.elastic.co/t/filebeat-not-creating-index-daily/243181/2 "2020-07-30T08:26:03Z")
Hi @daniel_fablius Could you share your output configuration, please? ---
**Author:** ![daniel\_fablius](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_fablius/32/67143_2.png) [@daniel\_fablius](https://discuss.elastic.co/u/daniel_fablius)\ **Post date:** [July 30, 2020, 8:37am UTC](https://discuss.elastic.co/t/filebeat-not-creating-index-daily/243181/3 "2020-07-30T08:37:32Z")
Hi @fgjensen i will show you my configMap yaml file instead ```auto apiVersion: v1 kind: ConfigMap metadata: name: filebeat-config namespace: logging labels: app: filebeat data: filebeat.yml: |- filebeat.config: modules: path: ${path.config}/modules.d/*.yml reload.enabled: true filebeat.autodiscover: providers: - type: kubernetes node: ${NODE_NAME} hints.enabled: true hints.default_config: type: container paths: - /var/log/containers/*${data.kubernetes.container.id}.log output.elasticsearch: hosts: '${ELASTICSEARCH_URL}' monitoring: enabled: true ``` i think the one that matters is the one ? ```auto output.elasticsearch: hosts: '${ELASTICSEARCH_URL}' ``` ---
**Author:** ![fgjensen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fgjensen/32/62320_2.png) [@fgjensen](https://discuss.elastic.co/u/fgjensen)\ **Post date:** [July 30, 2020, 8:52am UTC](https://discuss.elastic.co/t/filebeat-not-creating-index-daily/243181/4 "2020-07-30T08:52:26Z")
See you are deploying on a Kubernetes cluster 🙂 You could try to add an index filed to the output like this index: "filebeat-%{[agent.version]}-%{+yyyy.MM.dd}", see the documentation here: [https://www.elastic.co/guide/en/beats/filebeat/current/elasticsearch-output.html](https://www.elastic.co/guide/en/beats/filebeat/current/elasticsearch-output.html). Depending on your use case, you should also consider if you want a new index created every day. The recommended index size is around 50GB ---
**Author:** ![daniel\_fablius](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_fablius/32/67143_2.png) [@daniel\_fablius](https://discuss.elastic.co/u/daniel_fablius)\ **Post date:** [July 30, 2020, 9:08am UTC](https://discuss.elastic.co/t/filebeat-not-creating-index-daily/243181/5 "2020-07-30T09:08:22Z")
Yes, im deploying on Kubernetes cluster, my bad to not point it out in the beginning. so, i try to add the index field, and addition to that i add `setup.template.name and setup.template.pattern` also (based on this docs [https://www.elastic.co/guide/en/beats/filebeat/current/elasticsearch-output.html#index-option-es](https://www.elastic.co/guide/en/beats/filebeat/current/elasticsearch-output.html#index-option-es)) ```auto apiVersion: v1 kind: ConfigMap metadata: name: filebeat-config namespace: logging labels: app: filebeat data: filebeat.yml: |- filebeat.config: modules: path: ${path.config}/modules.d/*.yml reload.enabled: true filebeat.autodiscover: providers: - type: kubernetes node: ${NODE_NAME} hints.enabled: true hints.default_config: type: container paths: - /var/log/containers/*${data.kubernetes.container.id}.log output.elasticsearch: hosts: '${ELASTICSEARCH_URL}' setup.template.name: "filebeat" setup.template.pattern: "filebeat-*" monitoring: enabled: true ``` but after i re deploying my cluster, i still not seeing my 30th July (today) index.. ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/8/6864e09264b634301292401fff8f63a01931f7b4.png) \*ignore the face that it has 28 and 29 July, its created by ILM you mentioning about 50GB index size, is ILM what you means? if thats what you mean, i did try that, but since my cluster only live for office hours, the lifetime of an index is not 24 hours, so when i try ILM with 1 day Max age, it wont generate new index in the next day (since the cluster age is not 24 hours yet). so, is filebeat creating new index based on ILM only, or its creating new index by output configuration? since if its ILM, then i dont have any way to create it every dat then ---
**Author:** ![daniel\_fablius](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_fablius/32/67143_2.png) [@daniel\_fablius](https://discuss.elastic.co/u/daniel_fablius)\ **Post date:** [July 30, 2020, 9:37am UTC](https://discuss.elastic.co/t/filebeat-not-creating-index-daily/243181/6 "2020-07-30T09:37:29Z")
thanks for pointing out the docs @fgjensen, turns out i missed out that the ILM enabled by default, and it will disable the index settings in the output config ([https://www.elastic.co/guide/en/beats/filebeat/current/elasticsearch-output.html#index-option-es](https://www.elastic.co/guide/en/beats/filebeat/current/elasticsearch-output.html#index-option-es)) so i just need to disable it, and the index for today is created my last output config ```auto output.elasticsearch: hosts: '${ELASTICSEARCH_URL}' index: "filebeat-%{[agent.version]}-%{+yyyy.MM.dd}" setup.template.name: "filebeat" setup.template.pattern: "filebeat-*" setup.ilm.enabled: false ``` the index and setup.template\* have default value, so i suppose those actually not needed to be defined for me (since i use default value anyway) Thanks for help ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 27, 2020, 11:37am UTC](https://discuss.elastic.co/t/filebeat-not-creating-index-daily/243181/7 "2020-08-27T11:37:36Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.