Frequent cluster_block_exception Kibana logstash

[Sai_Avinash_Duddupud (avinash)]

Hi all,

I am getting the following errors in my logstash logs.

[INFO ] 2020-07-13 07:29:08.401 [[main]>worker0] elasticsearch - retrying failed action with response code: 429 ({"type"=>"cluster_block_exception", "reason"=>"index [blend] blocked by: [TOO_MANY_REQUESTS/12/index read-only / allow delete (api)];"})
[INFO ] 2020-07-13 07:29:08.402 [[main]>worker0] elasticsearch - retrying failed action with response code: 429 ({"type"=>"cluster_block_exception", "reason"=>"index [blend] blocked by: [TOO_MANY_REQUESTS/12/index read-only / allow delete (api)];"})
[INFO ] 2020-07-13 07:29:08.402 [[main]>worker0] elasticsearch - Retrying individual bulk actions that failed or were rejected by the previous bulk request. {:count=>2}

and after some searching, I found this link and used the below command and it worked well

curl -XPUT -H "Content-Type: application/json" https://[YOUR_ELASTICSEARCH_ENDPOINT]:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'

But again after 10 min or so, I started getting above errors in logstash and I hit the curl command again. this is happening very frequently.

Is there any permanent fix for this otherwise I had to hit the curl command many times every 20 min or so

[Christian_Dahlqvist]

Have a look at this blog post which discusses this problem and what causes it.

[Sai_Avinash_Duddupud (avinash)]

Came to know that there is low disk space....raised a request to increase the same....so hopefully no issues after that (but for now its running smooth)