Can't update disk space watermarks via API

Elastic Stack Elasticsearch
1

Hello there.
I'm trying to update thresholds of disk space in my ES cluster but can't do this due to error.

[admin@client1 ~]$ curl -X PUT "10.xxx.xxx.xxx:9200/_cluster/settings?flat_settings=true" -H 'Content- 
Type: application/json' -d'
{
"transient" : {
    "cluster.routing.allocation.disk.watermark.low" : "93%"
}, "persistent" : {
    "cluster.routing.allocation.disk.watermark.low" : "93%"
}
}
'
{"error":{"root_cause":[{"type":"remote_transport_exception","reason":"[Master1][10.xxx.xxx.xxx:9300][cluster:admin/settings/update]"}],"type":"illegal_argument_exception","reason":"low disk watermark [93%] more than high disk watermark [90%]"},"status":400}

The problem is that high disk watermark is already set to 95%. I checked elasticsearch.yml, persistent settings and transient settings. Everywhere high watermark is 95%.
The same problem occurs when i try to increase high watermark. It says that flood watermark is 95% but it's 98% in fact.

Where does these settings come from?

2

Question is still not solved :frowning:

3

Did you try to first increase cluster.routing.allocation.disk.watermark.high to 95% for example?

Note that you can disable watermark with cluster.routing.allocation.disk.threshold_enabled: false. But this is not recommended for production clusters.

You should really have more disk space. You can do that by adding more disk, adding more nodes or cleaning non needed data.

My 2 cents.

4

When I try to increase disk.watermark.high it says that flood_stage watermark (95%) is lower than new value of disk.watermark.high, but this is not true too.
I already set flood_stage watermark to 98%.

5

Did you try the second option I gave?

6

Are you talking about disabling watermarks? If yes, I didn't tried it.

7

Yes I do.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.