Since Kibana/Shield can't meet my req.s I shall build a separate UI that will use the Kibana REST API instead.
That or embedding Kibana visualizations/dashboards as iframes could be an option for a read-only view.
So, should each client have their own index?
I think this depends on what type of restrictions you need. If the data each client needs to see is separate from each other, you could put it in their own indexes. However, Shield supports document and field level restrictions. So if you have to share an index and segregate data by _type or some other attributes, you could use document level permissions to limit access to a subset of documents in an index.
If so, can myself as admin run queries on ALL the indexes?
Yes, that should be doable with Shield.
Does Shield have a REST API as well that I can use to ensure authentication and authorisation for a given client and access to their index?
In the current shipping version of Shield permissions are configured via config files, so you'd need config mgmt tools to automate it. In 5.0 we are adding a REST API to add users and manage roles within shieild.
Yes, we have our own OSS visualizations library, called vislib we are currently using.