Assigning ILM for metricbeat

I have started metricbeat used custome field.yml file to load template.
I have following in metricbeat.yml

setup.template.fields: "fields.yml"
setup.ilm.enabled: auto
setup.template.overwrite: true

now I started metricbeat on remote system and send data to logstash, it did created index
metricbeat-7.9.1 but didn't had any ILM assign. I am trying to assign ILM to that index and gives me this error. what am I doing wrong?

Hi!

From https://www.elastic.co/guide/en/beats/metricbeat/current/ilm.html#setup-ilm-option:

When auto (the default) is specified on version 7.0 and later, Metricbeat automatically uses index lifecycle management if the feature is enabled in Elasticsearch and has the required license; otherwise, Metricbeat creates daily indices.

Can you check if it could be a licence issue or if your Elasticsearch is properly configured?

yes I do have setup.ilm.enable:auto

this is running basic license.

I delete everything, recreated everything but still same. Data is coming from logstash and has following. It does create this index properly when I start logstash "metricbeat-7.9.1"

output {
elasticsearch {
hosts => ["elktst02:9200","elktst03:9200"]
index => "%{[@metadata][beat]}-%{[@metadata][version]}"
user => "elastic"
password => "changeme"
}
stdout { codec => json_lines }
}

and this is metricbeat.yml

setup.template.fields: "fields.yml"
setup.ilm.enabled: auto
setup.ilm.rollover_alias: "metricbeat-%{[agent.version]}"
setup.ilm.pattern: "{now/d}-000001"
setup.template.overwrite: true
setup.template.settings:
index.number_of_shards: 1
index.codec: best_compression

Did you run metricbeat setup? https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-template.html#load-template-manually

This will be needed in order to properly setup the indexes and ilms defined in Metricbeat config.

I did went through that whole document many time over. Need help telling me exactly what I need. getting very confused. what I need is
metricbeat name = metricbeat-%{[agent.version]}

and then when it rolls over it should be just metricbeat-%{[agent.version]}-yyyy.dd-00001

this is what I have in metricbeat.yml file

setup.template.fields: "fields.yml"
setup.ilm.enabled: auto
setup.ilm.rollover_alias: "metricbeat-%{[agent.version]}"
setup.ilm.pattern: "{+yyyy.MM}-000001"
setup.template.overwrite: true
setup.ilm.overwrite: true

and then from logstash I am creating index = metricbeat-7.9.3

Hi!

Running metricbeat setup --index-management -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["localhost:9200"]' with the proper hosts defined will do the setup.
I would suggest trying this incrementally so as to figure out what exactly configuration you need.
For example I would try with:

setup.template.name: "your_template_name"
setup.template.fields: "path/to/fields.yml"
setup.template.overwrite: true
setup.template.pattern: "some-*"

Then if you verify that it works you can add more settings like ilm etc.

C.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.