Jun 29 18:18:13 linux kibana: FATAL Error: Unable to complete saved object migrations for the [.kibana] index. Please check the health of your Elasticsearch cluster and try again. Unexpected Elasticsearch ResponseError: statusCode: 429, method: PUT, url: /.kibana_7.16.2_001/_mapping?timeout=60s error: [cluster_block_exception]: index [.kibana_7.16.2_001] blocked by: [TOO_MANY_REQUESTS/12/disk usage exceeded flood-stage watermark, index has read-only-allow-delete block];,
Jun 29 18:18:13 linux systemd: kibana.service: Main process exited, code=exited, status=1/FAILURE
Jun 29 18:18:13 linux systemd: kibana.service: Failed with result 'exit-code'.
Jun 29 18:18:16 linux systemd: kibana.service: Scheduled restart job, restart counter is at 53.
Jun 29 18:18:16 linux systemd: Stopped Kibana.
Jun 29 18:18:16 linux systemd: Started Kibana.
Your kibana is not running, the systemd service is constantly restarting it.
This is the reson:
The indice that kibana needs to write is marked as read-only because your elasticsearch cluster data path reached the flood-stage disk watermark, which means that the disk is 95% or more full, you need to clean up some space in the disk. If there is anything else running in the same server check if you can remove anything, if it is only for Elasticsearch you will need to delete some indices.