Elastic ILM Configuration

Elastic Stack Beats
,
1

I have filebeat writing directly into elastic and now need to configure ILM. My task is to do this through configuration so that it is deployable without user intervention.

Is there are documentation that details how to set up a configuration file?

2

Did you check the documentation in the official site?

3

Indeed, it however doesn't have that information that explains how to set up a file. What am I missing?

4

You do not setup an ILM with a file, you need to make a request to the API.

The documentation shows you how to do it using Kibana and also how the request should be if you do not want to use Kibana.

For example, this link on how to configure a lifecycle policy, click in the API example item to expand the code block and see the request.

You can also follow the link in that same page to the API documentation.

5

Thank you.

What do I need to do to get that api code run when deploying a new filebeat deployment?

6

Actually you are right and you can set this using a file as you already discovered the setting in your other post.

This answer in your other post is how the file needs to look like.

7

this is how you do it via filebeat. need to change filebeat.yml file once. run it.
let filebeat create all and then disable it.

output.elasticsearch.index: "yourindexname-*"
setup.template.name: "yourindexname"
setup.template.pattern: "yourindexname-*"

setup.template.fields: "/etc/filebeat/fields.yml"
setup.ilm.overwrite: true
setup.ilm.enabled: auto
setup.ilm.policy_name: "yourindexname"
setup.ilm.rollover_alias: "yourindexname-%{[agent.version]}"
setup.ilm.pattern: "{now/d}-000001"

What this will do is create IML name yourindexname ( you can adjust everything in it afterward)
it will create index pattern yourindexname-*
it will create index template yourindexname ( you can adjust number of shard, number of replica etc.. afterward if you like)
it will create index yourindexname--<today_date>-000001
and alias

yourindexname-<agent_version> ---> yourindexname-<filebeat version>-<today_date>-000001

and when your index will rollover it will create new index with
yourindexname--<today_date>-000002

and now your alias pointing to this index. and all that will be manage by ILM

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.