ELK 7.12.1 version ssl and tsl certificate issue

I have 3 nodes , in that 1 is master node and 2 are child node , while enable ssl and tsl certificate i am getting an error : below is the error.

at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) ~[?:?]
at org.elasticsearch.node.Node.(Node.java:571) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.node.Node.(Node.java:278) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:217) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:217) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:397) [elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) [elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) [elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:75) [elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:116) [elasticsearch-cli-7.12.1.jar:7.12.1]
at org.elasticsearch.cli.Command.main(Command.java:79) [elasticsearch-cli-7.12.1.jar:7.12.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) [elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:81) [elasticsearch-7.12.1.jar:7.12.1]
[2022-11-21T05:35:55,721][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [node1] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: you cannot specify a keystore and key file
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:75) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:116) ~[elasticsearch-cli-7.12.1.jar:7.12.1]
at org.elasticsearch.cli.Command.main(Command.java:79) ~[elasticsearch-cli-7.12.1.jar:7.12.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:81) ~[elasticsearch-7.12.1.jar:7.12.1]
Caused by: java.lang.IllegalArgumentException: you cannot specify a keystore and key file
at org.elasticsearch.xpack.core.ssl.CertParsingUtils.createKeyConfig(CertParsingUtils.java:223) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLConfiguration.createKeyConfig(SSLConfiguration.java:159) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLConfiguration.(SSLConfiguration.java:52) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.lambda$getSSLConfigurations$4(SSLService.java:504) ~[?:?]
at java.util.HashMap.forEach(HashMap.java:1425) ~[?:?]
at java.util.Collections$UnmodifiableMap.forEach(Collections.java:1521) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.getSSLConfigurations(SSLService.java:499) ~[?:?]
at org.elasticsearch.xpack.core.XPackPlugin.createSSLService(XPackPlugin.java:459) ~[?:?]
at org.elasticsearch.xpack.core.XPackPlugin.createComponents(XPackPlugin.java:292) ~[?:?]
at org.elasticsearch.node.Node.lambda$new$17(Node.java:567) ~[elasticsearch-7.12.1.jar:7.12.1]
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273) ~[?:?]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) ~[?:?]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) ~[?:?]
at org.elasticsearch.node.Node.(Node.java:571) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.node.Node.(Node.java:278) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:217) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:217) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:397) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.12.1.jar:7.12.1]

Please help me on this

below is my yml file --

Use a descriptive name for your cluster:

cluster.name: elk

xpack.security.transport.ssl.enabled: true
#xpack.security.transport.ssl.key: certs/node1.key
xpack.security.transport.ssl.certificate: certs/node1.crt
xpack.security.transport.ssl.certificate_authorities: certs/ca.crt
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/elastic-certificates.p12
xpack.security.transport.ssl.keystore.secure_password: Capgemini@1234
#xpack.security.transport.ssl.keystore.secure_password:
xpack.security.http.ssl.keystore.path: http.p12

xpack.security.enabled: true

xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: certs/node1.key
xpack.security.http.ssl.certificate: certs/node1.crt
xpack.security.http.ssl.certificate_authorities: certs/ca.crt
xpack.monitoring.collection.enabled: true

------------------------------------ Node ------------------------------------

Use a descriptive name for the node:

node.name: node1

Add custom attributes to the node:

#node.attr.rack: r1

node.master: true
node.data: true

----------------------------------- Paths ------------------------------------

Path to directory where to store the data (separate multiple locations by comma):

path.data: /var/lib/elasticsearch

Path to log files:

path.logs: /var/log/elasticsearch

Lock the memory on startup:

#bootstrap.memory_lock: true

Make sure that the heap size is set to about half the memory available

on the system and that the owner of the process is allowed to use this

limit.

Elasticsearch performs poorly when the system is swapping the memory.

---------------------------------- Network -----------------------------------

By default Elasticsearch is only accessible on localhost. Set a different

address here to expose this node on the network:

network.host: 172.16.0.62

By default Elasticsearch listens for HTTP traffic on the first free port it

finds starting at 9200. Set a specific HTTP port here:

http.port: 9200

--------------------------------- Discovery ----------------------------------

Pass an initial list of hosts to perform discovery when this node is started:

The default list of hosts is ["127.0.0.1", "[::1]"]

discovery.seed_hosts: ["node1.elastic.test.com","node2.elastic.test.com","node3.elastic.test.com"]

Bootstrap the cluster using an initial set of master-eligible nodes:

cluster.initial_master_nodes: ["node1.elastic.test.com"]

For more information, consult the discovery and cluster formation module documentation.

---------------------------------- Various -----------------------------------

Require explicit names when deleting indices:

#action.destructive_requires_name: true

Hello! The stack trace mentions this error:

org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: you cannot specify a keystore and key file

I think the error is about these two settings which cannot be set at the same time.

xpack.security.http.ssl.keystore.path: http.p12
xpack.security.http.ssl.key: certs/node1.key

In general, a Keystore contains a PrivateKeyEntry which wraps two things:

  • PrivateKey
  • Certificate Chain

If you specify a key file containing a PrivateKey, then you are specifying a PrivateKey twice. Only one of the key vs keystore settings can be used.

I have commented bellow two lines
#xpack.security.http.ssl.keystore.path: http.p12
#xpack.security.http.ssl.key: certs/node1.key

Still i am getting the below error

org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: you cannot specify a keystore and key file
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:75) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:116) ~[elasticsearch-cli-7.12.1.jar:7.12.1]
at org.elasticsearch.cli.Command.main(Command.java:79) ~[elasticsearch-cli-7.12.1.jar:7.12.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:81) ~[elasticsearch-7.12.1.jar:7.12.1]
Caused by: java.lang.IllegalArgumentException: you cannot specify a keystore and key file
at org.elasticsearch.xpack.core.ssl.CertParsingUtils.createKeyConfig(CertParsingUtils.java:223) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLConfiguration.createKeyConfig(SSLConfiguration.java:159) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLConfiguration.(SSLConfiguration.java:52) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.lambda$getSSLConfigurations$4(SSLService.java:504) ~[?:?]
at java.util.HashMap.forEach(HashMap.java:1425) ~[?:?]
at java.util.Collections$UnmodifiableMap.forEach(Collections.java:1521) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.getSSLConfigurations(SSLService.java:499) ~[?:?]
at org.elasticsearch.xpack.core.XPackPlugin.createSSLService(XPackPlugin.java:459) ~[?:?]
at org.elasticsearch.xpack.core.XPackPlugin.createComponents(XPackPlugin.java:292) ~[?:?]
at org.elasticsearch.node.Node.lambda$new$17(Node.java:567) ~[elasticsearch-7.12.1.jar:7.12.1]
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273) ~[?:?]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) ~[?:?]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) ~[?:?]
at org.elasticsearch.node.Node.(Node.java:571) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.node.Node.(Node.java:278) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:217) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:217) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:397) ~[elasticsearch-7.12.1.jar:7.12.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.12.1.jar:7.12.1]

I think this still refers to keystores, so it should be removed. Also, passwords should go in elasticsearch-keystore.

I tried starting 7.12.1 with your config. I had to tweak the settings, and I added some comments. I got past the error you saw using elasticsearch.yml and elasticsearch-keystore similar to this.

elasticsearch.yml example outline (change the file paths)

cluster.name: elk
bootstrap.memory_lock: true
xpack.security.enabled: true
xpack.monitoring.collection.enabled: true

# Transport/TLS protocol settings
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate

# Transport key/cert settings, do not mix with keystore/truststore settings due to overlap
xpack.security.transport.ssl.key: ~/example/elasticsearch-7.12.1/config/certs/transport-node1.key
xpack.security.transport.ssl.certificate: ~/example/elasticsearch-7.12.1/config/certs/transport-node1.crt
xpack.security.transport.ssl.certificate_authorities: [ ~/example/elasticsearch-7.12.1/config/certs/transport-ca.crt ]

# Transport keystore/truststore settings, do not mix with key/cert settings due to overlap
#xpack.security.transport.ssl.keystore.path: ~/example/elasticsearch-7.12.1/config/certs/transport-node1-keystore.p12
#xpack.security.transport.ssl.truststore.path: ~/example/elasticsearch-7.12.1/config/certs/transport-node1-truststore.p12

# HTTPS protocol settings
xpack.security.http.ssl.enabled: true

# HTTP key/cert settings, do not mix with Keystore/Truststore settings due to overlap
xpack.security.http.ssl.key: ~/example/elasticsearch-7.12.1/config/certs/http-node1.key
xpack.security.http.ssl.certificate: ~/example/elasticsearch-7.12.1/config/certs/http-node1.crt
xpack.security.http.ssl.certificate_authorities: [ ~/example/elasticsearch-7.12.1/config/certs/http-ca.crt ]

# HTTP keystore/truststore settings, do not mix with key/cert settings due to overlap
#xpack.security.http.ssl.keystore.path: ~/example/elasticsearch-7.12.1/config/certs/http-node1-keystore.p12
#xpack.security.http.ssl.truststore.path: ~/example/elasticsearch-7.12.1/config/certs/http-node1-truststore.p12

elasticsearch-keystore example commands (change the passwords)

bin/elasticsearch-keystore create

# Only use these settings if you chose key/cert settings in elasticsearch.yml
#echo "HTTP-PrivateKey-Password"        | bin/elasticsearch-keystore add xpack.security.http.ssl.secure_key_passphrase
#echo "Transport-PrivateKey-Password"   | bin/elasticsearch-keystore add xpack.security.transport.ssl.secure_key_passphrase

# Only use these settings if you chose keystore/truststore settings in elasticsearch.yml
echo "HTTP-Keystore-Password"        | bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password
echo "HTTP-Truststore-Password"      | bin/elasticsearch-keystore add xpack.security.http.ssl.truststore.secure_password
echo "Transport-Keystore-Password"   | bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
echo "Transport-Truststore-Password" | bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password

I have generated crt.pem and key.pem by using the below command
openssl pkcs12 -in elastic-certificates.p12 -out newfile.crt.pem -clcerts -nokeys
openssl pkcs12 -in elastic-certificates.p12 -out newfile.key.pem -nocerts -nodes

But for ssl.certificate_authorities i need to generate ca.crt for filebeat
Could you please help me with command to install ca.crt

It depends how you ran elasticsearch-certutil. I am going to assume you ran elasticsearch-certutil twice like so:

bin/elasticsearch-certutil ca
bin/elasticsearch-certutil cert -ca elastic-stack-ca.p12

I ran the two commands you provided, and made a copy of the first one to export the CA cert from the first P12.

openssl pkcs12 -in elastic-stack-ca.p12 -out ca.crt.pem -clcerts -nokeys
openssl pkcs12 -in elastic-certificates.p12 -out newfile.crt.pem -clcerts -nokeys
openssl pkcs12 -in elastic-certificates.p12 -out newfile.key.pem -nocerts -nodes

I ended up with these files:

elastic-certificates.p12
elastic-stack-ca.p12
ca.crt.pem
newfile.crt.pem
newfile.key.pem