# Why I am getting one huge metricbeat index instead of daily indices?
**URL:** https://discuss.elastic.co/t/why-i-am-getting-one-huge-metricbeat-index-instead-of-daily-indices/252789
**Category:** Beats
**Tags:** metricbeat
**Created:** [October 21, 2020, 7:38am UTC](https://discuss.elastic.co/t/why-i-am-getting-one-huge-metricbeat-index-instead-of-daily-indices/252789 "2020-10-21T07:38:06Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@elasticheart](https://discuss.elastic.co/u/elasticheart)
#### Post date: [October 21, 2020, 7:38am UTC](https://discuss.elastic.co/t/why-i-am-getting-one-huge-metricbeat-index-instead-of-daily-indices/252789/1 "2020-10-21T07:38:06Z")
Hi,
I am using elastic stack 7.9.2. I have set up metricbeat and it is exporting to elasticsearch. I haven't modified the index name setting in `metricbeat.yml` and is taking the default setting `index: "metricbeat-%{[agent.version]}-%{+yyyy.MM.dd}"`.
What I expected was metricbeat index getting created daily, like `metricbeat-7.9.2-2020.10.1` , `metricbeat-7.9.2-2020.10.2` , `metricbeat-7.9.2-2020.10.3` etc. Also, I was thinking about creating some ILM policy, so that indices older than 5 days will get automatically deleted.
Now, when I take a look at the indices, I can see one huge index `metricbeat-7.9.2-2020.10.14-000001` (started metricbeat on 14th). Why its happening like this and how can I fix this, to achieve what I am looking for?
Thanks.
---
### Author:  [@borna\_talebi](https://discuss.elastic.co/u/borna_talebi)
#### Post date: [October 21, 2020, 8:00am UTC](https://discuss.elastic.co/t/why-i-am-getting-one-huge-metricbeat-index-instead-of-daily-indices/252789/2 "2020-10-21T08:00:06Z")
Hi,
In your `metricbeat.yml` file did you disabled ILM using `setup.ilm.enabled: false`?
Because if you didn't, metricbeat will ignore` index: "metricbeat-%{[agent.version]}-%{+yyyy.MM.dd}"` and use indices with numbers at the end to manage the lifecycle.
according to docs:
> When [index lifecycle management (ILM)](https://www.elastic.co/guide/en/beats/filebeat/current/ilm.html) is enabled, the default `index` is `"filebeat-%{[agent.version]}-%{+yyyy.MM.dd}-%{index_num}"` , for example, `"filebeat-7.9.2-2020-10-21-000001"` . Custom `index` settings are ignored when ILM is enabled.
---
### Author:  [@elasticheart](https://discuss.elastic.co/u/elasticheart)
#### Post date: [October 21, 2020, 8:09am UTC](https://discuss.elastic.co/t/why-i-am-getting-one-huge-metricbeat-index-instead-of-daily-indices/252789/3 "2020-10-21T08:09:10Z")
Hi @borna_talebi , below is my ILM config
```
# index name.
#setup.ilm.enabled: auto
#setup.ilm.rollover_alias: 'metricbeat'
#setup.ilm.pattern: "{now/d}-000001"
#setup.ilm.policy_name: "mypolicy"
#setup.ilm.policy_file:
setup.ilm.check_exists: false
#setup.ilm.overwrite: false
```
I have just disabled `setup.ilm.check_exists` and everything else is default. Is there anything need to be modified?
---
### Author:  [@borna\_talebi](https://discuss.elastic.co/u/borna_talebi)
#### Post date: [October 21, 2020, 8:15am UTC](https://discuss.elastic.co/t/why-i-am-getting-one-huge-metricbeat-index-instead-of-daily-indices/252789/4 "2020-10-21T08:15:24Z")
> [@elasticheart](#):
>
> Is there anything need to be modified?
It depends. if you want to fix your huge index then don't change anything and just configure your ILM and it should fix it.
But if you want to have daily indices then use `setup.ilm.enabled: false`
---
### Author:  [@elasticheart](https://discuss.elastic.co/u/elasticheart)
#### Post date: [October 21, 2020, 8:50am UTC](https://discuss.elastic.co/t/why-i-am-getting-one-huge-metricbeat-index-instead-of-daily-indices/252789/5 "2020-10-21T08:50:54Z")
@borna_talebi I got it and thanks for your suggestion 👍
Can you advice me how to configure ILM, say if I just want 1 index `metricbeat`, but I don't want messages older than 5 days in it. What should I do to achieve this? I don't want to archive old indices, say I don't want `warm` and `cold`. I just want the messages older than 5 days to get deleted. Also, I would like to know at what time this policy apply on the indices, say, the message on 01-10-2020 get deleted at 12'o clock on 16-10-2020?
---
### Author:  [@borna\_talebi](https://discuss.elastic.co/u/borna_talebi)
#### Post date: [October 21, 2020, 9:26am UTC](https://discuss.elastic.co/t/why-i-am-getting-one-huge-metricbeat-index-instead-of-daily-indices/252789/6 "2020-10-21T09:26:08Z")
> [@elasticheart](#):
>
> but I don't want messages older than 5 days in it
I'm not sure if this is the best solution but here is my suggestion:
1. In hot phase, set rollover to 1 day so ilm would create daily indices.
2. In delete phase, set `Timing for delete phase` to `5 days from rollover`.
Just to be clear, ILM works with **indices** not **messages** (documents).
---
### Author:  [@elasticheart](https://discuss.elastic.co/u/elasticheart)
#### Post date: [October 21, 2020, 9:39am UTC](https://discuss.elastic.co/t/why-i-am-getting-one-huge-metricbeat-index-instead-of-daily-indices/252789/7 "2020-10-21T09:39:48Z")
Ok. Lemme try out this and I will reply here 👍
---
### Author:  [@elasticheart](https://discuss.elastic.co/u/elasticheart)
#### Post date: [November 6, 2020, 7:15am UTC](https://discuss.elastic.co/t/why-i-am-getting-one-huge-metricbeat-index-instead-of-daily-indices/252789/8 "2020-11-06T07:15:19Z")
I did whatever need to be, still no hope. Lemme explain;
**I have these many indices** 👆
**And this is my metricbeat index I am concerned about** 👆
  
**This is my ILM policy** 👆
**These are my templates** 👆
 
**And this is the template explained** 👆
Below is my ILM policy explained.
```
PUT _ilm/policy/metricbeat
{
"policy": {
"phases": {
"hot": {
"min_age": "0ms",
"actions": {
"rollover": {
"max_age": "1d",
"max_size": "1gb"
},
"set_priority": {
"priority": null
}
}
},
"warm": {
"actions": {
"set_priority": {
"priority": null
}
}
},
"delete": {
"min_age": "2d",
"actions": {
"delete": {
"delete_searchable_snapshot": true
}
}
}
}
}
}
```
As per the policy, the index should be max 1 GB size and 1 day old. This index was created yesterday and more than 1 GB size (pri 1.1GB, total 2.1GB). How to fix this?
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 4, 2020, 9:15am UTC](https://discuss.elastic.co/t/why-i-am-getting-one-huge-metricbeat-index-instead-of-daily-indices/252789/9 "2020-12-04T09:15:27Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.