Problem solved. It was a hostname verification problem. In my elasticsearch.yml, I changed the _xpack.ssl.verification_mode option (default to full). First, I tried:
xpack.ssl.verification_mode: certificate
And it worked. The documentation is not really clear on this option, but I guess it just skips the hostname verification when enabling TLS.
Then, I put it back to full, and I set the network.host option (I guess it is the one used in the hostname verification ?)
xpack.ssl.verification_mode: full # that's the default value
network.host: 127.0.0.1 # IP of my node
And then I could start elastic search with my single node, with the TLS node-to-node option enabled, without any errors. I also tried with 2 nodes and it had the same behavior. Handshake failed because of the hostname verification. But, when I set network.host "properly", everything worked fine.
I am still not sure about what is going on behind all this (why is TLS failing when I have only one node ? what "hostname" is compared to the certificate CN during the handshake ? And I'm a beginner with networks)... but the problem is solved.
Thanks for your help.