Elasticsearch not responding for CAT and other apis

The fundamental issue is that Elasticsearch is writing a (tiny) file for each shard every 30 seconds, and your system seem to be unable to cope with this workload. This is particularly problematic on nodes holding too many shards; I've never seen this on a cluster with under 300 shards like yours, and suspect there's something wrong with your disks if they are struggling under this kind of load.

The fix I linked was to avoid doing these updates if nothing changed. I can't think of a workaround apart from upgrading, or using faster disks.