ILM setup dynamic alias in filebeat

Elastic Stack Beats
2

You are right, you cannot setup dynamic alias for ILM.

One option if you want to load ilm through filebeat - create static alias name by indexing all modules data into single index and add the dynamic value as an additional field (eg: fileset.module) in the index document, then you can apply the filter on the consumption side.

Otherwise, you can load the ilm policies and templates through Elastic API externally.
For loading externally you can follow the below approach

  1. If you have a limited set of values for "fileset.module" values, then create a separate template for each one with separate alias

eg:

PUT _template/my_template {
"index_patterns": ["filebeat-7.1.1-apache-*"],
"settings": {
"number_of_shards": 1,
"number_of_replicas": 1,
"index.lifecycle.name": "filebeat-7.1.1-apache-policy",
"index.lifecycle.rollover_alias": "filebeat-7.1.1-apache"
}
}

filebeat-7.1.1-apache-policy - ILM Policy name

PUT _template/my_template {
"index_patterns": ["filebeat-7.1.1-audit-*"],
"settings": {
"number_of_shards": 1,
"number_of_replicas": 1,
"index.lifecycle.name": "filebeat-7.1.1-audit-policy",
"index.lifecycle.rollover_alias": "filebeat-7.1.1-audit"
}
}

Then bootstrap the process by creating first index

#PUT /<filebeat-7.1.1-apache-{now/d}-1> with URI encoding:

PUT /%3Cfilebeat-7.1.1-apache-%7Bnow%2Fd%7D-1%3E {
"aliases": {
"filebeat-7.1.1-apache":{
"is_write_index": true
} } }

Then in your filebeat conf..
index: "filebeat-7.1.1-%{[fileset.module]}"
Note: As I said assuming you setup templates/ilm policies for all combinations of "fileset.module" values.

3 Likes