ILM hot-warm with beats and component templates

Elastic Stack Elasticsearch
1

I've been looking at implementing hot-warm architecture in the most default way possible, but with 7.8 the component templates were introduced which changes some behavior previously documented in guides for this topic.

I want to make sure all my new indicies created by any beat (filebeat, metricbeat, auditbeat and packetbeat) is allocated to my hot nodes on creation, then moved to warm nodes according to my ILM policy. In the most recent guides I can find ( https://www.elastic.co/blog/implementing-hot-warm-cold-in-elasticsearch-with-index-lifecycle-management ) one would use "Multiple Template Matching" to just add the allocation settings together with the default beat template index, using a second template matching the same index patterns, which would allow one to use both the index template and ILM policy created by the beat - but with component templates this seems to be deprecated.

Using Multiple Template Matching also gives a deprecation warning that confirms this:

# PUT filebeat-7.8.0-2020.07.15-000001
#! Deprecation: index [filebeat-7.8.0-2020.07.15-000001] matches multiple legacy templates [filebeat-7.8.0, filebeat-hot-warm-cold], composable templates will only match a single template

As I want to avoid any custom templates for as much as possible, how would I go about and allocate new indices to my "hot" nodes without having to replace the default templates?

1 Like
2

So after some digging in the github issues I can only assume that there will be a new way of doing this and that beats will provide component templates that one can use for the same scenario. Couldn't find more than "we should look at this and make sure it works".

So to answer my own question: For now one will have to do it the old way and live with the deprecation warnings (or simply don't rely on the beats managed templates) - future updates will probably, hopefully, address these changes and provide new documentation and guides to achieve a hot-warm(-cold) architecture.

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.