# Watermark floodstage not stopping usage of diskspace

**URL:** <https://discuss.elastic.co/t/watermark-floodstage-not-stopping-usage-of-diskspace/245415>\
**Category:** Elasticsearch\
**Created:** [August 18, 2020, 12:38pm UTC](https://discuss.elastic.co/t/watermark-floodstage-not-stopping-usage-of-diskspace/245415 "2020-08-18T12:38:09Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Darshan\_Kamat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/darshan_kamat/32/44993_2.png) [@Darshan\_Kamat](https://discuss.elastic.co/u/Darshan_Kamat)\
**Post date:** [August 18, 2020, 12:38pm UTC](https://discuss.elastic.co/t/watermark-floodstage-not-stopping-usage-of-diskspace/245415/1 "2020-08-18T12:38:09Z")

</div>

I have 1 node cluster where flood stage watermark is breached but it seems ES is still occupying data .Is this correct ?

PFB my logs :

> [2020-05-30T06:18:44,774][WARN][o.e.c.r.a.DiskThresholdMonitor] [elastic\_226] flood stage disk watermark [95%] exceeew][elastic\_226][/apps/efk/elasticsearch-7.6.0/data/nodes/0] free: 21.8mb[0%], all indices on this node will be marke  
> [2020-05-30T06:19:14,775][WARN][o.e.c.r.a.DiskThresholdMonitor] [elastic\_226] flood stage disk watermark [95%] exceeew][elastic\_226][/apps/efk/elasticsearch-7.6.0/data/nodes/0] free: 18.4mb[0%], all indices on this node will be marke  
> [2020-05-30T06:19:44,776][WARN][o.e.c.r.a.DiskThresholdMonitor] [elastic\_226] flood stage disk watermark [95%] exceeew][elastic\_226][/apps/efk/elasticsearch-7.6.0/data/nodes/0] free: 15mb[0%], all indices on this node will be marked  
> [2020-05-30T06:20:14,777][WARN][o.e.c.r.a.DiskThresholdMonitor] [elastic\_226] flood stage disk watermark [95%] exceeew][elastic\_226][/apps/efk/elasticsearch-7.6.0/data/nodes/0] free: 11.6mb[0%], all indices on this node will be marke  
> [2020-05-30T06:20:44,779][WARN][o.e.c.r.a.DiskThresholdMonitor] [elastic\_226] flood stage disk watermark [95%] exceeew][elastic\_226][/apps/efk/elasticsearch-7.6.0/data/nodes/0] free: 8.2mb[0%], all indices on this node will be marked  
> [2020-05-30T06:21:14,780][WARN][o.e.c.r.a.DiskThresholdMonitor] [elastic\_226] flood stage disk watermark [95%] exceeew][elastic\_226][/apps/efk/elasticsearch-7.6.0/data/nodes/0] free: 4.8mb[0%], all indices on this node will be marked  
> [2020-05-30T06:21:44,781][WARN][o.e.c.r.a.DiskThresholdMonitor] [elastic\_226] flood stage disk watermark [95%] exceeew][elastic\_226][/apps/efk/elasticsearch-7.6.0/data/nodes/0] free: 1.5mb[0%], all indices on this node will be mark

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 18, 2020, 12:58pm UTC](https://discuss.elastic.co/t/watermark-floodstage-not-stopping-usage-of-diskspace/245415/2 "2020-08-18T12:58:38Z")

</div>

The flood stage causes Elasticsearch to make indices read-only and redistribute shards if it can. It does not delete any data.

---

<div class="post-metadata">

**Author:** ![Darshan\_Kamat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/darshan_kamat/32/44993_2.png) [@Darshan\_Kamat](https://discuss.elastic.co/u/Darshan_Kamat)\
**Post date:** [August 18, 2020, 1:22pm UTC](https://discuss.elastic.co/t/watermark-floodstage-not-stopping-usage-of-diskspace/245415/3 "2020-08-18T13:22:08Z")

</div>

So when u say indices are read only , does it mean new data and new indices wont be created ?  
Also how do i limit ES from utilizing more disk space like 95% max .Isnt that what "flood stage watermark" supposed to do ?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 18, 2020, 1:28pm UTC](https://discuss.elastic.co/t/watermark-floodstage-not-stopping-usage-of-diskspace/245415/4 "2020-08-18T13:28:17Z")

</div>

Yes. No new indices will be created and no new data indexed.

---

<div class="post-metadata">

**Author:** ![Darshan\_Kamat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/darshan_kamat/32/44993_2.png) [@Darshan\_Kamat](https://discuss.elastic.co/u/Darshan_Kamat)\
**Post date:** [August 18, 2020, 1:30pm UTC](https://discuss.elastic.co/t/watermark-floodstage-not-stopping-usage-of-diskspace/245415/5 "2020-08-18T13:30:45Z")

</div>

So in this case how come disk availability is getting decreased continously even after watermark breach all the way to 0 % is my question

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 18, 2020, 1:39pm UTC](https://discuss.elastic.co/t/watermark-floodstage-not-stopping-usage-of-diskspace/245415/6 "2020-08-18T13:39:36Z")

</div>

Do you have anything else writing there, e.g. logs?

---

<div class="post-metadata">

**Author:** ![Darshan\_Kamat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/darshan_kamat/32/44993_2.png) [@Darshan\_Kamat](https://discuss.elastic.co/u/Darshan_Kamat)\
**Post date:** [August 18, 2020, 1:56pm UTC](https://discuss.elastic.co/t/watermark-floodstage-not-stopping-usage-of-diskspace/245415/7 "2020-08-18T13:56:14Z")

</div>

No..it has only elastic search..but as I mentioned I do have only 1 node cluster...could that be the cause of the problem?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 15, 2020, 1:56pm UTC](https://discuss.elastic.co/t/watermark-floodstage-not-stopping-usage-of-diskspace/245415/8 "2020-09-15T13:56:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
