Move old indices on hhd

Elastic Stack Elasticsearch
1

Hi,

We have some nodes with ssd storages and store indices per month.
The plan is to add some data nodes and move old indices (e.g. 2 years old) to this new nodes with hdd storage.
How to perform this movement (in detail) ?
Can Elasticsearch automatically check if the indices are older than 2 years and move them?

Regards
Andi

2

To do this you will want to look at Shard Allocation Filtering in the following link: https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-allocation.html#shard-allocation-filtering

The main idea here is you can tag your nodes with specific attributes (in your case maybe disk_type:ssd and disk_type:hdd and then in your index settings set the attributes which are allowed for the shards of that index (i.e. set which nodes the shards can be allocated to). This setting is dynamic so you can change it on a live index and Elasticsearch will move the shards (if necessary) to conform to the new allocation rules without any downtime for that index.

Also, for automating this I would recommend using the Elasticsearch Curator tool which was specifically built for managing time-series indices in Elasticsearch: https://github.com/elastic/curator

The documentation about using Curator to change allocation settings can be found in the Curator docs here: https://www.elastic.co/guide/en/elasticsearch/client/curator/current/allocation.html

1 Like