Thank you for your reply.
It is now working. I had already read the link you said but
"If hostname verification fails, you can disable this verification by setting xpack.security.http.ssl.verification_mode to certificate."
which is what is needed if you don't have a SAN.
However in end I set xpack.security.http.ssl.enabled: false previously it was true. This was set to true i.e following point one of the trouble shooting, because there were times when the set-password tried to use http.
I have set the passwords, but still not 100% sure why it is now working.