"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})

[alexus]

it looks like I ran out of space. I free up some space by deleted bunch of unneeded indices and docs.

... yet still keep getting following error:

logstash | [2018-12-30T15:38:10,978][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})

I even tried running following query as well

PUT /_settings
{
  "index": {
    "blocks": {
      "read_only_allow_delete": "true"
    }
  }
}

Please advise.

[Christian_Dahlqvist]

How much disk space did you free up? How much is currently used?

[alexus]

~75GB (total), ~54GB (used), ~20GB (free) / 73%

I just checked, not all data got deleted and now whenever I run same query I now get following message:

"reason": "blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"

[Christian_Dahlqvist]

Did you reset the read-only index block after you freed up space?

[alexus]

I reset read-only for entire cluster:

PUT /_settings
{
  "index.blocks.read_only_allow_delete": null
}