# Enlarge Cluster with new nodes **URL:** \ **Category:** Elasticsearch\ **Created:** [February 11, 2021, 9:21am UTC](https://discuss.elastic.co/t/enlarge-cluster-with-new-nodes/263974 "2021-02-11T09:21:59Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![cadirol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cadirol/32/38459_2.png) [@cadirol](https://discuss.elastic.co/u/cadirol)\ **Post date:** [February 11, 2021, 9:21am UTC](https://discuss.elastic.co/t/enlarge-cluster-with-new-nodes/263974/1 "2021-02-11T09:21:59Z")
Hi all We are planing to enlarge our cluster with 16 nodes in the end. the first 16 nodes are 4 years old, but still healthy and good enough! But as you can imagine, 4 years ago, we added 4TB Disk space in each node. now, we can do the same - 4TB per node - but from my point of view, this is not the way to go. We need current hardware. I know from several postings, different disk sizes are not the way to go! What is a good way? > **[Index-level shard allocation filtering | Elasticsearch Reference \[7.11\] |...](https://www.elastic.co/guide/en/elasticsearch/reference/current/shard-allocation-filtering.html)** allocation filtering? Or i can build the new nodes with, let's say 8TB and create partitions with only 4TB. As long as the Old nodes are fine, we can add on this nodes 8TB as well and increasing the storage server by server. Thank you very much for your input! BR A ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [February 14, 2021, 11:57pm UTC](https://discuss.elastic.co/t/enlarge-cluster-with-new-nodes/263974/2 "2021-02-14T23:57:14Z")
Ultimately Elasticsearch will balance by shard count, and disk size. Allocation filtering will work, you could make the new ones cold nodes to store more data. ---
**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Post date:** [February 15, 2021, 3:50am UTC](https://discuss.elastic.co/t/enlarge-cluster-with-new-nodes/263974/3 "2021-02-15T03:50:42Z")
if I were you then I will just use new node with faster disk as data only node. add them to cluster and slowly remove existing node function from data to master or cold storage only. I even will add 4x1tb SSD in a node rather then one 4tb disk. ( permitting disk allocation allowed in hardware) ---
**Author:** ![cadirol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cadirol/32/38459_2.png) [@cadirol](https://discuss.elastic.co/u/cadirol)\ **Post date:** [February 15, 2021, 8:37am UTC](https://discuss.elastic.co/t/enlarge-cluster-with-new-nodes/263974/4 "2021-02-15T08:37:58Z")
Hi @warkolm Thank you vor your reply. I think i will use the Allocation Filter based on hostname. so i can define which index will be stores on which nodes. If we plan to upgrade the storage from the "older" nodes, i can set the shard allocation (on index settings) on a per node basis, where the shards are stored ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [February 15, 2021, 10:35pm UTC](https://discuss.elastic.co/t/enlarge-cluster-with-new-nodes/263974/5 "2021-02-15T22:35:33Z")
You should probably abstract that to a different level, otherwise that's a lot of management. ---
**Author:** ![cadirol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cadirol/32/38459_2.png) [@cadirol](https://discuss.elastic.co/u/cadirol)\ **Post date:** [February 16, 2021, 10:28am UTC](https://discuss.elastic.co/t/enlarge-cluster-with-new-nodes/263974/6 "2021-02-16T10:28:28Z")
What do you mean with "abstract that to a different level"? Yes, with allocation filter based on Hostname, i have to sett the right hostname in each index Settings... Can be scripted, but yes, a lot of work ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [February 16, 2021, 9:20pm UTC](https://discuss.elastic.co/t/enlarge-cluster-with-new-nodes/263974/7 "2021-02-16T21:20:48Z")
Why not just use tags like `big_disk` and `small_disk` and use it that way. ---
**Author:** ![cadirol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cadirol/32/38459_2.png) [@cadirol](https://discuss.elastic.co/u/cadirol)\ **Post date:** [February 18, 2021, 7:18am UTC](https://discuss.elastic.co/t/enlarge-cluster-with-new-nodes/263974/8 "2021-02-18T07:18:59Z")
You mean on old nodes `node.attr.size: small_disk` and on new nodes `node.attr.size: big_disks` At the end, my node configuration (elasticsearch.yml) should look like this? ```auto node.attr.rack: north node.attr.size: small_disks node.attr.box_type: hot ``` But in any case, i have to set theese _small\_disk and big\_disk_ settings on all indices. Right? According to the documentation i have to set it like this, individually on all indices: ```auto PUT test/_settings { "index.routing.allocation.require.size": "small_disks" "index.routing.allocation.require.rack": "north" "index.routing.allocation.require.box_type": "hot" } ``` I hope i understand your suggestion correct. Thank you very much ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [February 18, 2021, 9:21pm UTC](https://discuss.elastic.co/t/enlarge-cluster-with-new-nodes/263974/9 "2021-02-18T21:21:28Z")
You may want to merge the concept of small and hot, but that's up to you. Yes you need to add the allocation tags to the indices. Use index templates, or even better, [ILM][ILM: Manage the index lifecycle | Elasticsearch Reference [7.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html)) to do this for you. ---
**Author:** ![cadirol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cadirol/32/38459_2.png) [@cadirol](https://discuss.elastic.co/u/cadirol)\ **Post date:** [February 19, 2021, 8:57am UTC](https://discuss.elastic.co/t/enlarge-cluster-with-new-nodes/263974/10 "2021-02-19T08:57:32Z")
ILM sound interesting, i have to take a look on it! As we have hot (NVME Disks) and Warm (SATA) Disks in the current cluster i have to merge the "box size" to it. The idea behind, to use node name for the index filter allocator was, that i have to change only the settings in each index if a "small\_disk" node would be upgraded to a "big\_disk" node. But anyway, if a Disk upgrade occur, i ha to drain the node, replace hardware and restart. In this Process, the node attribute can be changed as well... Thank you very much for your inputs! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 19, 2021, 8:57am UTC](https://discuss.elastic.co/t/enlarge-cluster-with-new-nodes/263974/11 "2021-03-19T08:57:55Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.