When an elastic index moves into a read only state, it can prevent additional data being written to the cluster. Typically a read only state occurs when an index in on a node that reaches 85% disk space used.
The cluster does have an API to get details for each index, so it may be possible to scan each index and check if it's in read only state.
- how to know if one or more indexes are marked as read only, would you please let me know?
- how can we generate an alert somewhere - likely grafana - when this happens
Note: we use cerebro ES admin tool as well