How to setup external access to my Azure VM running Kibana on port 5601?

That looks good to me. Kibana should be able to communicate with ES via the local connection.