TOO_MANY_REQUESTS/12/disk usage exceeded

Recently credentials are failing in the Kibana web interface. Checking the logs there is an error indicating disk usage is exceeded.

{"type":"log","@timestamp":"2021-08-08T20:41:52-05:00","tags":["error","plugins","apm"],"pid":2366,"message":"Could not create index: '.apm-custom-link'. Error: cluster_block_exception: [cluster_block_exception] Reason: index [.apm-custom-link] blocked by: [TOO_MANY_REQUESTS/12/disk usage exceeded flood-stage watermark, index has read-only-allow-delete block];."}

I've seen different references around on the net stating to "free up space" and then run these commands to unlock the indices.

curl -XPUT -H "Content-Type: application/json" http://10.6.67.30:9200/_cluster/settings -d '{ "transient": { "cluster.routing.allocation.disk.threshold_enabled": false } }'
curl -XPUT -H "Content-Type: application/json" http://10.6.67.30:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'

How do I go about safely deleting indices via the CLI since I cannot use the Kibana interface?

Thanks,

Check out Delete index API | Elasticsearch Guide [7.14] | Elastic

The question asked for how to safely delete indices when you are unable to access the Kibana interface. Your answer points to a process that requires access to the Kibana interface to work.

What CMD/CLI options (if any) are available for deleting indices when Kibana won't start due to 'disk usage exceeded' notifications? I am having the same issue this morning - Kibana won't start - I am seeing the same in my terminal - I am using ELK on Windows, so CMD prompt options would also be helpful.

In that doc link you can see the Copy as curl and that will let you run something directly from a shell, without the need for Kibana.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.