This one is very complicated as I don't see much success on getting any insight here or on internet. Ever one just points to Elasticsearch document. which is for me complicated as I am not guy who understands security stuff clearly.
Spend few hours to test this out and here is the procedure if anyone ever had a this problem.
Two command and copy file to appropriate location
It is simple three step process. First to create authorities ca keystore .p12 file
then use that to create certificate .p12 file
first step
as root user.
# /usr/share/elasticsearch/bin/elasticsearch-certutil ca --days 1000
....
....
...
Please enter the desired output file [elastic-stack-ca.p12]:
Enter password for elastic-stack-ca.p12 : (hit enter if you don't need password here)
this will create elastic-stack-ca.p12 in /usr/share/elasticsearch/ directory, which is your ES_HOME
second step
Now use following command to use that ca file and create new certificate
# /usr/share/elasticsearch/bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 --days 1000
......
.....
....
Enter password for CA (elastic-stack-ca.p12) :
Please enter the desired output file [elastic-certificates.p12]: elastic-certificates.p12
Enter password for elastic-certificates.p12 :
Certificates written to /usr/share/elasticsearch/elastic-certificates.p12
This file should be properly secured as it contains the private key for
your instance.
This file is a self contained file and can be copied and used 'as is'
For each Elastic product that you wish to configure, you should copy
this '.p12' file to the relevant configuration directory
and then follow the SSL configuration instructions in the product guide.
For client applications, you may only need to copy the CA certificate and
configure the client to trust this certificate.
third step
This has to be done with all cluster node down or you can do it in rolling fashion
put this last file /elastic-certificates.p12 at proper location. whereever your old file was present
and apply this setting to elasticsearch.yml and start elasticsearch daemon
# my old file was elastic-certificates_certi.p12
# my new file is elastic-certificates.p12
#xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/config/elastic-certificates_certi.p12
#xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/config/elastic-certificates_certi.p12
xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/config/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/config/elastic-certificates.p12
I confirm this works. I have tested this on small container setup first. then apply same logic to my cluster and all is good working.