# Metricbeat ILM rollover issue

**URL:** <https://discuss.elastic.co/t/metricbeat-ilm-rollover-issue/246063>\
**Category:** Beats\
**Tags:** ilm-index-lifecycle-management, metricbeat\
**Created:** [August 24, 2020, 7:25am UTC](https://discuss.elastic.co/t/metricbeat-ilm-rollover-issue/246063 "2020-08-24T07:25:36Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![ragmenon](https://avatars.discourse-cdn.com/v4/letter/r/ba8739/32.png) [@ragmenon](https://discuss.elastic.co/u/ragmenon)\
**Post date:** [August 24, 2020, 7:25am UTC](https://discuss.elastic.co/t/metricbeat-ilm-rollover-issue/246063/1 "2020-08-24T07:25:36Z")

</div>

Hi, My metric dashboard in kibana crashed every now and then in 2 days with error

##### Error

Internal Server Error (500)

##### URL

[https://kibana.integ.tvld.tech/api/metrics/snapshot](https://kibana.integ.tvld.tech/api/metrics/snapshot)

## i suspect this might an issue with metric beat ILM configuration, i hjave configured ILM with life cycle jsonon metric.yml

```auto
metricbeat.config.modules:
  path: "${path.config}/modules.d/*.yml"
  reload.enabled: 'false'
setup.template.settings:
  index.number_of_shards: '1'
  index.codec: best_compression
logging.level: info
logging.to_files: 'true'
logging.files:
  path: "/var/log/metricbeat"
  name: metricbeat
  keepfiles: '5'
  permissions: '0644'
setup.kibana:
  host: http://parkbni01.infra.tvld.tech:8080
output.elasticsearch:
  hosts:
  - https://parelki03.infra.tvld.tech:9200
  - https://parelki02.infra.tvld.tech:9200
  - https://parelki01.infra.tvld.tech:9200
  protocol: https
  username: tvld_metricbeat
  password: wacAIrAurw/FcK1juPfKR5dtzfXDway2DnrUc+DbbNE=
setup.ilm.policy_name: metricbeat-rollover-10d
setup.ilm.policy_file: "/etc/metricbeat/lifecycle.json"

```

## lifecycle.json

```auto
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": {
            "max_size": "10gb",
            "max_age": "10d"
          },
          "set_priority": {
            "priority": 100
          }
        }
      },
      "warm": {
        "min_age": "5d",
        "actions": {
          "set_priority": {
            "priority": 50
          }
        }
      },
      "cold": {
        "min_age": "10d",
        "actions": {
          "set_priority": {
            "priority": 0
          }
        }
      },
      "delete": {
        "min_age": "15d",
        "actions": {
          "delete": {

          }
        }
      }
    }
  }
}

```

While index and index template is created it works fine. but after 2 days the iLM policy defined dissappears from the metrics.  
anything i miss for config.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 24, 2020, 8:23am UTC](https://discuss.elastic.co/t/metricbeat-ilm-rollover-issue/246063/2 "2020-08-24T08:23:35Z")

</div>

> [@ragmenon](#):
>
> Internal Server Error (500)

Please share the Kibana log.

---

<div class="post-metadata">

**Author:** ![ragmenon](https://avatars.discourse-cdn.com/v4/letter/r/ba8739/32.png) [@ragmenon](https://discuss.elastic.co/u/ragmenon)\
**Post date:** [August 24, 2020, 12:40pm UTC](https://discuss.elastic.co/t/metricbeat-ilm-rollover-issue/246063/4 "2020-08-24T12:40:18Z")

</div>

Hi this is error i get on kibana,

```auto
Aug 24 08:46:52 parkbni01 kibana: {"type":"error","@timestamp":"2020-08-24T06:46:52Z","tags":[],"pid":73650,"level":"error","error":{"message":"Internal Server Error","name":"Error","stack":"Error: Internal Server Error\n at HapiResponseAdapter.toError (/usr/share/kibana/src/core/server/http/router/response_adapter.js:132:19)\n at HapiResponseAdapter.toHapiResponse (/usr/share/kibana/src/core/server/http/router/response_adapter.js:86:19)\n at HapiResponseAdapter.handle (/usr/share/kibana/src/core/server/http/router/response_adapter.js:81:17)\n at Router.handle (/usr/share/kibana/src/core/server/http/router/router.js:160:34)\n at process._tickCallback (internal/process/next_tick.js:68:7)"},"url":{"protocol":null,"slashes":null,"auth":null,"host":null,"port":null,"hostname":null,"hash":null,"search":null,"query":{},"pathname":"/api/metrics/snapshot","path":"/api/metrics/snapshot","href":"/api/metrics/snapshot"},"message":"Internal Server Error"} 

```

To resolve this error i have to stop the metricbeat and remove the index and templates and new created index will have ILM config with lifecycle.json.

---

<div class="post-metadata">

**Author:** ![ragmenon](https://avatars.discourse-cdn.com/v4/letter/r/ba8739/32.png) [@ragmenon](https://discuss.elastic.co/u/ragmenon)\
**Post date:** [August 25, 2020, 4:52am UTC](https://discuss.elastic.co/t/metricbeat-ilm-rollover-issue/246063/5 "2020-08-25T04:52:27Z")

</div>

@warkolm  
WIll you be able to help 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 22, 2020, 4:52am UTC](https://discuss.elastic.co/t/metricbeat-ilm-rollover-issue/246063/6 "2020-09-22T04:52:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.