Heartbeat - Setting a custom ILM policy

Elastic Stack Beats
1

Hi all,

I'm having some problems trying to set the ILM policy of heartbeat indices.
I'm running heartbeat with the following configuration (heartbeat.yml):

heartbeat.config.monitors:
  path: ${path.config}/monitors.d/*.yml
  reload.enabled: true
  reload.period: 5s

setup.template.settings:
  index.number_of_shards: 1
  index.index.number_of_replicas: 0
  index.codec: best_compression

output.elasticsearch:
  hosts: ["localhost:9200"]

processors:
  - add_host_metadata: ~

setup.ilm.enabled: true
setup.ilm.rollover_alias: "heartbeat"
setup.ilm.pattern: "{now/d}-000001" 
setup.ilm.policy.name: "custom_policy"
setup.ilm.policy.file: "/<path-to-file>/policy.json"
setup.ilm.overwrite: true

I'm using a custom ilm policy which is provided to heartbeat by file. This is the content of policy.json

{
  "policy": {
    "phases": {
      "delete": {
        "min_age": "1d",
        "actions": {
          "delete": {} 
        }
      }
    }
  }
}

Now, when I launch heartbeat I get the following messages in the heartbeat logs

INFO    [index-management]      idxmgmt/std.go:252      Auto ILM enable success.
INFO    [index-management.ilm]  ilm/std.go:134  do not generate ilm policy: exists=true, overwrite=false
INFO    [index-management]      idxmgmt/std.go:265      ILM policy successfully loaded.
INFO    [index-management]      idxmgmt/std.go:394      Set setup.template.name to '{heartbeat-7.3.1 {now/d}-000001}' as ILM is enabled.
INFO    [index-management]      idxmgmt/std.go:399      Set setup.template.pattern to 'heartbeat-7.3.1-*' as ILM is enabled.
INFO    [index-management]      idxmgmt/std.go:433      Set settings.index.lifecycle.rollover_alias in template to {heartbeat-7.3.1 {now/d}-000001} as ILM is enabled.
INFO    [index-management]      idxmgmt/std.go:437      Set settings.index.lifecycle.name in template to {heartbeat-7.3.1 {"policy":{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_size":"50gb"}}}}}}} as ILM is enabled.
INFO    template/load.go:88     Template heartbeat-7.3.1 already exists and will not be overwritten.
INFO    [index-management]      idxmgmt/std.go:289      Loaded index template.
INFO    [index-management]      idxmgmt/std.go:300      Write alias successfully generated.

As you can see, it says that a template already exists (the default heartbeat template) and that it will not be overwritten.
It seems like it has not updated the default ILM policy.

In fact, if I check the ilm policy through the _ilm/explain endpoint

GET heartbeat*/_ilm/explain

I get the response:

{
  "indices" : {
    "heartbeat-7.3.1-2019.10.04-000001" : {
      "index" : "heartbeat-7.3.1-2019.10.04-000001",
      "managed" : true,
      "policy" : "heartbeat-7.3.1",
      "lifecycle_date_millis" : 1570188426186,
      "phase" : "hot",
      "phase_time_millis" : 1570188426236,
      "action" : "complete",
      "action_time_millis" : 1570188426220,
      "step" : "complete",
      "step_time_millis" : 1570188426236,
      "phase_execution" : {
        "policy" : "heartbeat-7.3.1",
        "phase_definition" : {
          "min_age" : "0ms",
          "actions" : { }
        },
        "version" : 5,
        "modified_date_in_millis" : 1570188387183
      }
    }
  }
}

which is not the ilm policy I defined, but the default one.

However I explicitly set the property setup.ilm.overwrite: true in the configuration file, so I don't understand why it doesn't change the ilm policy in the index settings.

Am I doing something wrong?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.