Hi, did reach out to support for guidance, and the root cause was that I'd done monitoring alert setup on the actual production Kibana... But instead I needed to setup Kibana on the dedicated monitor cluster. Makes sense I guess thinking about it.
Also I had to enable the nodes with role 'remote_cluster_client' to get things working fully.
All good now! Alerts setup with PagerDuty actions as well so we're cookin' with fire now.