Invalid configuration for xpack.security.transport.ssl

Elastic Stack Elasticsearch
1

We tried to update to ES 8.0 deb package. ES refused to start with an error

[2022-02-23T09:10:15,109][ERROR][o.e.b.Bootstrap          ] [mon] Exception
org.elasticsearch.ElasticsearchSecurityException: invalid configuration for xpack.security.transport.ssl - [xpack.security.transport.ssl.enabled] is not set, but the following settings have
 been configured in elasticsearch.yml : [xpack.security.transport.ssl.keystore.secure_password,xpack.security.transport.ssl.truststore.secure_password]
        at org.elasticsearch.xpack.core.ssl.SSLService.validateServerConfiguration(SSLService.java:652) ~[?:?]
        at org.elasticsearch.xpack.core.ssl.SSLService.loadSslConfigurations(SSLService.java:612) ~[?:?]
        at org.elasticsearch.xpack.core.ssl.SSLService.<init>(SSLService.java:156) ~[?:?]
        at org.elasticsearch.xpack.core.XPackPlugin.createSSLService(XPackPlugin.java:471) ~[?:?]
        at org.elasticsearch.xpack.core.XPackPlugin.createComponents(XPackPlugin.java:312) ~[?:?]
        at org.elasticsearch.node.Node.lambda$new$16(Node.java:662) ~[elasticsearch-8.0.0.jar:8.0.0]
        at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273) ~[?:?]
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) ~[?:?]
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921) ~[?:?]
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
        at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) ~[?:?]
        at org.elasticsearch.node.Node.<init>(Node.java:676) ~[elasticsearch-8.0.0.jar:8.0.0]
        at org.elasticsearch.node.Node.<init>(Node.java:277) ~[elasticsearch-8.0.0.jar:8.0.0]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:234) ~[elasticsearch-8.0.0.jar:8.0.0]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:234) ~[elasticsearch-8.0.0.jar:8.0.0]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:358) [elasticsearch-8.0.0.jar:8.0.0]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:166) [elasticsearch-8.0.0.jar:8.0.0]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:157) [elasticsearch-8.0.0.jar:8.0.0]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:77) [elasticsearch-8.0.0.jar:8.0.0]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112) [elasticsearch-cli-8.0.0.jar:8.0.0]
        at org.elasticsearch.cli.Command.main(Command.java:77) [elasticsearch-cli-8.0.0.jar:8.0.0]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:122) [elasticsearch-8.0.0.jar:8.0.0]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80) [elasticsearch-8.0.0.jar:8.0.0]

In Elasticsearch.yml there is NOT ssh enabled or password set. Where does this come from?

The only settings we added is path.data and

thread_pool.search.queue_size: 10000
xpack.ml.enabled: false

# maximum shards setting
cluster.max_shards_per_node: 270

But even original Elasticsearch.yml file is failing to start.

2

These settings are in the Elasticsearch keystore (located in /etc/elastisearch/elasricsearch.keystore)

You can remove them with /usr/share/elastisearch/bin/elastisearch-keystore remove <name-of-the-setting> to unblock you for now. (Don’t do that if you don’t know the passwords and you need them because you’re going to lose them - you can use /usr/share/elastisearch/bin/elastisearch-keystore show <name-of-the-setting> to get the value of the setting before removing.)

But it’s important to figure out how you ended up in the situation in the first place. What version where you upgrading from? How did you do the upgrade? Did you have security configured and you wanted to disable it now ?

1 Like
3

Yes, the keyword really exists. We just did update from 7.8 version from debian package. Nothing else, no security before.

root@a:~# /usr/share/elasticsearch/bin/elasticsearch-keystore list
autoconfiguration.password_hash
keystore.seed
xpack.security.http.ssl.keystore.secure_password
xpack.security.transport.ssl.keystore.secure_password
xpack.security.transport.ssl.truststore.secure_password
4

That’s unfortunate and looks like a bug. Just to verify, can you share the exact commands and steps that you ran during the upgrade ? Did you use apt or dpkg? We need this in order to reproduce the behaviour and fix any issues.
Also, did you ever start the 7.8 cluster before the upgrade ?

For what it's worth, I cannot replicate this behavior that you are seeing, neither with apt nor with dpkg. Also, note that a direct upgrade from 7.8 to 8.0 is not supported ( for unrelated reasons ), see our docs in Upgrade Elasticsearch | Elasticsearch Guide [8.0] | Elastic

Thanks !

5

The validation that prevents startup if there are SSL settings but no value for *.ssl.enabled is new in 8.0, so it's possible that the underlying causes were present in 7.x

There was a deprecation warning, but I don't recall if it existed in 7.8 - which is part of the reason why it is important to upgrade to 7.17 and run the upgrade assistant before moving to 8.0

1 Like
6

The fact that the above exists in the keystore leads me to believe that autoconfiguration ran and that the settings we see in the keystore are the auto-generated ones. But if this happened, there should also be configuration to enable TLS in elastisearch.yml so I’m not entirely sure how / why @Akaren got to this state.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.