ES node storage size mixing?

Elastic Stack Elasticsearch
1

We have a few year old cluster of 15 servers (2 ES nodes per server) with 8TBs of local storage each, but are we running low on space and having to cut back on retention. I want to add 80TBs to increase our total storage from 120TBs to 200TBs, but we are limited on physical rack space, so is it possible to add 4 servers with 20TBs of space each? (all numbers usable space after hardware RAID)

I've heard that Elasticsearch likes each ES node to be configured with the same amount of storage, is this true? I couldn't locate any documentation that explicitly states this. Would ES balance the shards ok across the node storage differences, or would 12 TBs be "wasted" on the new servers?

2

You can easily do this if you setup hot/warm/cold tiering using ILM. Where your nodes with more storage would be cold nodes.

3

Unfortunately I don't think that would work for us. We have a lot of hot data, and we're not allowed to keep it past a certain amount of time, so we don't really have any cold data.

4

If you want to think of it another way - standard and dense node types. Then you can store more data on the dense ones over the standard ones.

Hot/warm/cold is a logical construct that you can repurpose for this type of thing too.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.