# Having issues with low watermark / disk space **URL:** \ **Category:** Elasticsearch\ **Created:** [February 24, 2025, 8:45am UTC](https://discuss.elastic.co/t/having-issues-with-low-watermark-disk-space/374934 "2025-02-24T08:45:14Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![cybertron](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cybertron/32/141591_2.png) [@cybertron](https://discuss.elastic.co/u/cybertron)\ **Post date:** [February 24, 2025, 8:45am UTC](https://discuss.elastic.co/t/having-issues-with-low-watermark-disk-space/374934/1 "2025-02-24T08:45:14Z")
Hi I have been struggling with low watermark that keeps getting hit on some nodes in my cluster. I have a 6 node cluster, that unfortunatly have different diskspace. 3 nodes have 5 TB and 3 nodes have 10TB. The problem is that the three nodes with less disk hits the low watermark, which in turn stops upgrades and patches from automatically install and update. the three nodes with large disks has more than enoght disk space. Is there any setting i can set to fix this? If I raise the disk space, it only moves more indices from other nodes down to the "low disk" nodes. i have tried changing the watermarks, but it always just crosses the threshold anyway. so if I set low watermark to 20%, it will fill up to 19.9%. , if I set to 300GB -\> it fills upp to 298GB and so on... so whatever the value, it always fills it up just enought to cross the threshold. ---
**Author:** ![cybertron](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cybertron/32/141591_2.png) [@cybertron](https://discuss.elastic.co/u/cybertron)\ **Post date:** [February 24, 2025, 8:50am UTC](https://discuss.elastic.co/t/having-issues-with-low-watermark-disk-space/374934/2 "2025-02-24T08:50:05Z")
```auto shards disk.indices disk.used disk.avail disk.total disk.percent host 524 6.4tb 6.4tb 3.2tb 9.6tb 66 node_3 370 4.3tb 4.3tb 468.1gb 4.8tb 90 node_1 523 7tb 7tb 2.6tb 9.6tb 72 node_4 524 6.7tb 6.8tb 2.8tb 9.6tb 70 node_5 310 4.3tb 4.3tb 485.7gb 4.8tb 90 node_2 341 4.3tb 4.3tb 481.9gb 4.8tb 90 node_1 ``` ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [February 24, 2025, 8:55am UTC](https://discuss.elastic.co/t/having-issues-with-low-watermark-disk-space/374934/3 "2025-02-24T08:55:53Z")
> [@cybertron](#): > > The problem is that the three nodes with less disk hits the low watermark, This is normal, especially in a cluster with varying disk space. See [these docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-cluster.html#disk-based-shard-allocation), particularly the bit that says > It is normal for nodes to temporarily exceed the high watermark from time to time. > [@cybertron](#): > > which in turn stops upgrades and patches from automatically install and update. > the three nodes with large disks has more than enoght disk space. This is not normal. There's no reason to avoid upgrading just because a node has exceeded the low watermark. ---
**Author:** ![cybertron](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cybertron/32/141591_2.png) [@cybertron](https://discuss.elastic.co/u/cybertron)\ **Post date:** [February 24, 2025, 8:58am UTC](https://discuss.elastic.co/t/having-issues-with-low-watermark-disk-space/374934/4 "2025-02-24T08:58:46Z")
ok, thanks for the fast reply. it says that when low watermark is use no new indices can be created. The reason I ask is that I am going to upgrade this cluster to 8.17 soon (running 7.17.25 now) and the upgrade assistant say that it is critical to fix this. ```auto Disk usage exceeds low watermark, which will prevent reindexing indices during upgrade. Get disk usage on all nodes below the value specified in cluster.routing.allocation.disk.watermark.low (nodes impacted: ``` ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [February 24, 2025, 9:23am UTC](https://discuss.elastic.co/t/having-issues-with-low-watermark-disk-space/374934/5 "2025-02-24T09:23:08Z")
Hmm ok I see, I think that's a bug in the upgrade assistant. If _all_ nodes were over the low watermark then this'd be true, but as long as there are some nodes with free disk space then that should be fine I think. Would you report the bug? ---
**Author:** ![cybertron](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cybertron/32/141591_2.png) [@cybertron](https://discuss.elastic.co/u/cybertron)\ **Post date:** [February 24, 2025, 9:45am UTC](https://discuss.elastic.co/t/having-issues-with-low-watermark-disk-space/374934/6 "2025-02-24T09:45:24Z")
oh that would explain it! great. I'll report it