ILM: Routing data based on AWS availability zones

Elastic Stack Elasticsearch
1

We currently have hot and warm nodes spread around 3 availability zones (AZ) in AWS.

I want to start using ILM, and I'd like to know how to configure the data transfer between warm and hot nodes so that hot nodes in an AZ always sends the data to warm nodes in the same AZ.

This is because AWS charges for moving data between AZs, but not when we move data within an AZ.

Is it possible to do this using ILM?

Thanks.

2

Hi dwilches,

ILM can move data based on node attributes, so as long as the nodes have different attributes assigned to them within an AZ, then you would be able to configure ILM actions to move the data between different nodes.

I would recommend checking out https://www.elastic.co/blog/implementing-hot-warm-cold-in-elasticsearch-with-index-lifecycle-management for an example of how to do it (you don't necessarily have to use "hot", "warm", or "cold" for attributes.

3

Thanks!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.