Recommended specs for Master and Coordinating nodes in a 80TB, 8B docs cluster

Elastic Stack Elasticsearch
1

We have a 21 node ES 6.3 cluster with a Hot-Warm-Cold architecture and we would like to move to have three dedicated Master nodes for this cluster and add some Coordinating nodes. As stated in the title the cluster is stable at around 80TB, because we drop older indices. We hope to mitigate some problems we've had with master election during recovery and increase search performance. Are there any recommendations of what specs these machines need?

Information about current nodes:
Hot: 4 nodes
48 cores
256GB memory
5TB SSD

Warm: 12 nodes
48 cores
200GB memory
7TB SSD

Cold: 5 nodes
40 cores
256GB memory
6TB HDD

2

Dedicated master nodes typically do not need a lot of resources. Can you provide the full output of the cluster stats API so we can get an idea what the cluster looks like, especially with regard to number of indices and shards in the cluster?

3

Indices: 721
Shards: 7090
Cluster stats API output: https://gist.github.com/highpie/3badfe792424a86d99582c0cf16dc937
Any other recommendations about the current cluster are also greatly appreciated!