Shard allocation with Hot warm architecture and Vertical scaling?

Elastic Stack Elasticsearch
1

Hi All,

Just curious how do you route shards from hot to warm nodes and prevent same shard allocation on a physical server in a Vertical scaling model where each physical servers

  • 2 or more hot ES nodes with data on SSD
  • 2 or more warm ES nodes with data on HDD

It's a combination of https://www.elastic.co/blog/hot-warm-architecture and https://www.elastic.co/elasticon/conf/2016/sf/scaling-elasticsearch-at-netsuite-good-bad-and-ugly

I've used cluster.routing.allocation.same_shard.host which worked for some indexes but failed for others because shards on hot ES nodes refused to move to warm ES nodes on the same physical server because they think they are on the same physical server (which is actually true). However, what I want to prevent is that same shards (primary and replica) should not allocate on hot nodes or on warm nodes on the same physical server, yet allow shards to move from hot to warm as if they are on a separate physical server.

I believe https://www.elastic.co/guide/en/elasticsearch/reference/2.3/allocation-awareness.html is the solution for such deployment, but would appreciate more inputs from the community.

Thanks

2

https://www.elastic.co/blog/hot-warm-architecture is the best option (which is what the second link also uses).

cluster.routing.allocation.same_shard.host is for when you are running multiple nodes per host, it's not for hot/warm.

3

Since I have 2 physical servers, each with 48 core CPU, 256 GB RAM, 2x SSD 1 TB, 24 TB HDD, I run hot/warm + vertical scaling model with hot indexes on SSD and warm indexes on HDD. In that way, CPU and RAM won't be wasted.

4

Ahh ok.