Thanks for the reply Mark.
I guess we will have to rely on search preferences (eg, _prefer_nodes:xxx) to direct queries to the SSD nodes, and use shard allocation awareness to distinguish the spinning disk nodes from the SSD nodes, so the SSD nodes always have a complete copy of the index (albeit made up of both primary and replica shards).
Would that work? Are there any other options we can look at?