# How to delete .monitoring\*mb\* indices automatically after period of time?

**URL:** https://discuss.elastic.co/t/how-to-delete-monitoring-mb-indices-automatically-after-period-of-time/264647
**Category:** Beats
**Tags:** elastic-stack-monitoring, metricbeat
**Created:** [February 18, 2021, 6:18am UTC](https://discuss.elastic.co/t/how-to-delete-monitoring-mb-indices-automatically-after-period-of-time/264647 "2021-02-18T06:18:14Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![dabo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dabo/32/95229_2.png) [@dabo](https://discuss.elastic.co/u/dabo)
#### Post date: [February 18, 2021, 6:18am UTC](https://discuss.elastic.co/t/how-to-delete-monitoring-mb-indices-automatically-after-period-of-time/264647/1 "2021-02-18T06:18:14Z")

</div>

Hi,

We are using Cloud Platinum subscription.  
We enabled Metricbeat monitoring on 21st of January.

Seems like .monitoring indices (old and new) are deleted only after 1 month or not deleted at all.

Can we control retention period on mb monitoring system indices?  
Are we supposed to set up ILM policies (and add it to system index templates) to control rollover and deletion of these system indices?

Please note that we have `xpack.monitoring.history.duration` set to "3d" which is not working. Is that related only to legacy monitoring and not to metribeat?

`GET _cluster/settings?filter_path=*.xpack.monitoring&include_defaults=true`

```auto
    {
      "persistent" : {
        "xpack" : {
          "monitoring" : {
            "collection" : {
              "enabled" : "true",
              "interval" : "10s"
            },
            "exporters" : {
              " __no-default-local__" : {
                "type" : "local",
                "enabled" : "false"
              }
            }
          }
        }
      },
      "defaults" : {
        "xpack" : {
          "monitoring" : {
            "collection" : {
              "cluster" : {
                "stats" : {
                  "timeout" : "10s"
                }
              },
              "node" : {
                "stats" : {
                  "timeout" : "10s"
                }
              },
              "indices" : [],
              "ccr" : {
                "stats" : {
                  "timeout" : "10s"
                }
              },
              "enrich" : {
                "stats" : {
                  "timeout" : "10s"
                }
              },
              "index" : {
                "stats" : {
                  "timeout" : "10s"
                },
                "recovery" : {
                  "active_only" : "false",
                  "timeout" : "10s"
                }
              },
              "ml" : {
                "job" : {
                  "stats" : {
                    "timeout" : "10s"
                  }
                }
              }
            },
            "history" : {
              "duration" : "3d"
            },
            "elasticsearch" : {
              "collection" : {
                "enabled" : "true"
              }
            },
            "enabled" : "true"
          }
        }
      }
    }

```

Any help is appreciated.

Kind regards,  
Dane

---

<div class="post-metadata">

### Author: ![dabo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dabo/32/95229_2.png) [@dabo](https://discuss.elastic.co/u/dabo)
#### Post date: [February 18, 2021, 8:20am UTC](https://discuss.elastic.co/t/how-to-delete-monitoring-mb-indices-automatically-after-period-of-time/264647/2 "2021-02-18T08:20:09Z")

</div>

I also have some questions about `.ds-logs-elastic_agent.filebeat-default.*` indices which are larger and occupy a **Hot** data node.  
These system indices are by default set to `"_tier_preference": "data_warm,data_hot"`.

- Why are these indices located in **Hot** node when we also have **Warm** nodes available?
- How are templates for these indices named?
- And also the same question for them about retention period.

Thanks,  
Dane

---

<div class="post-metadata">

### Author: ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)
#### Post date: [February 18, 2021, 2:02pm UTC](https://discuss.elastic.co/t/how-to-delete-monitoring-mb-indices-automatically-after-period-of-time/264647/3 "2021-02-18T14:02:51Z")

</div>

Welcome @dabo!

How many ES clusters are involved here? Is it a single one, or are you using a dedicated cluster for monitoring data?

---

<div class="post-metadata">

### Author: ![dabo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dabo/32/95229_2.png) [@dabo](https://discuss.elastic.co/u/dabo)
#### Post date: [February 18, 2021, 2:19pm UTC](https://discuss.elastic.co/t/how-to-delete-monitoring-mb-indices-automatically-after-period-of-time/264647/4 "2021-02-18T14:19:15Z")

</div>

Hi Chris,

Thanks! 🙂

Currently we have only one cluster for all.

---

<div class="post-metadata">

### Author: ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)
#### Post date: [February 18, 2021, 3:16pm UTC](https://discuss.elastic.co/t/how-to-delete-monitoring-mb-indices-automatically-after-period-of-time/264647/5 "2021-02-18T15:16:25Z")

</div>

> [@dabo](#):
>
> ```auto
> " __no-default-local__" : {
> "type" : "local",
> "enabled" : "false"
> }
> 
> ```

I think this is the problem. We have automated curation of the `.monitoring-*` indices through the `xpack.monitoring.history.duration` cluster setting, but it requires local exporter enabled.

Did you explicitly disable this local exporter? I'd expect it to be enabled by default

---

<div class="post-metadata">

### Author: ![dabo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dabo/32/95229_2.png) [@dabo](https://discuss.elastic.co/u/dabo)
#### Post date: [February 18, 2021, 4:15pm UTC](https://discuss.elastic.co/t/how-to-delete-monitoring-mb-indices-automatically-after-period-of-time/264647/6 "2021-02-18T16:15:54Z")

</div>

No, didn't change anything explicitly.

I was thinking the same if that should be enabled, but thought it is only for legacy monitoring, not for monitoring via Metricbeat...?

If we are not exporting data back to our cluster where is the data we see in Stack Monitoring coming from?

---

<div class="post-metadata">

### Author: ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)
#### Post date: [February 18, 2021, 8:17pm UTC](https://discuss.elastic.co/t/how-to-delete-monitoring-mb-indices-automatically-after-period-of-time/264647/7 "2021-02-18T20:17:26Z")

</div>

You're right, it's confusing. This will change soon, but for now, you'll need to enable the local exporter to kick off the curator task in Elasticsearch which respects the `xpack.monitoring.history.duration` cluster setting. Currently, ILM is not used to curate monitoring indices.

---

<div class="post-metadata">

### Author: ![dabo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dabo/32/95229_2.png) [@dabo](https://discuss.elastic.co/u/dabo)
#### Post date: [February 19, 2021, 6:41am UTC](https://discuss.elastic.co/t/how-to-delete-monitoring-mb-indices-automatically-after-period-of-time/264647/8 "2021-02-19T06:41:33Z")

</div>

Great, thanks! We will try that.

Will that setting affect the .ds-\* indices as well? These are having must greater impact on our disk at the moment.

**UPDATE** : found an answer to all my questions about .ds-\* indices.

Thank you for your help, you can close the topic.

Kind regards,  
Dane

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 19, 2021, 8:41am UTC](https://discuss.elastic.co/t/how-to-delete-monitoring-mb-indices-automatically-after-period-of-time/264647/9 "2021-03-19T08:41:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
