When `discovery.type: single-node` is not set, disabling TLS at the HTTP layer will cause elasticsearch to fail to start

linghengqian · October 14, 2024, 7:37am

When discovery.type: single-node is not set, disabling TLS at the HTTP layer will cause elasticsearch to fail to start.
I used a simple docker-compose.yml to verify this. Just run docker compose up -d.

services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.15.2
    environment:
      xpack.security.http.ssl.enabled: false

There are a lot of errors similar to the following in the docker container.

[2] bootstrap checks failed. You must address the points described in the following [2] lines before starting Elasticsearch. For more information see [https://www.elastic.co/guide/en/elasticsearch/reference/8.15/bootstrap-checks.html]
bootstrap check failure [1] of [2]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured; for more information see [https://www.elastic.co/guide/en/elasticsearch/reference/8.15/_discovery_configuration_check.html]
bootstrap check failure [2] of [2]: Transport SSL must be enabled if security is enabled. Please set [xpack.security.transport.ssl.enabled] to [true] or disable security by setting [xpack.security.enabled] to [false]; for more information see [https://www.elastic.co/guide/en/elasticsearch/reference/8.15/bootstrap-checks-xpack.html#bootstrap-checks-tls] | @timestamp=2024-10-14T07:30:21.612Z log.level=ERROR ecs.version=1.2.0 service.name=ES_ECS event.dataset=elasticsearch.server process.thread.name=main log.logger=org.elasticsearch.bootstrap.Elasticsearch elasticsearch.node.name=75fc863d6b9c elasticsearch.cluster.name=docker-cluster 
ERROR: Elasticsearch did not exit normally - check the logs at /usr/share/elasticsearch/logs/docker-cluster.log
stopping ... | @timestamp=2024-10-14T07:30:21.618Z log.level=INFO ecs.version=1.2.0 service.name=ES_ECS event.dataset=elasticsearch.server process.thread.name=elasticsearch-shutdown log.logger=org.elasticsearch.node.Node elasticsearch.node.name=75fc863d6b9c elasticsearch.cluster.name=docker-cluster 
stopped | @timestamp=2024-10-14T07:30:21.639Z log.level=INFO ecs.version=1.2.0 service.name=ES_ECS event.dataset=elasticsearch.server process.thread.name=elasticsearch-shutdown log.logger=org.elasticsearch.node.Node elasticsearch.node.name=75fc863d6b9c elasticsearch.cluster.name=docker-cluster 
closing ... | @timestamp=2024-10-14T07:30:21.639Z log.level=INFO ecs.version=1.2.0 service.name=ES_ECS event.dataset=elasticsearch.server process.thread.name=elasticsearch-shutdown log.logger=org.elasticsearch.node.Node elasticsearch.node.name=75fc863d6b9c elasticsearch.cluster.name=docker-cluster 
closed | @timestamp=2024-10-14T07:30:21.649Z log.level=INFO ecs.version=1.2.0 service.name=ES_ECS event.dataset=elasticsearch.server process.thread.name=elasticsearch-shutdown log.logger=org.elasticsearch.node.Node elasticsearch.node.name=75fc863d6b9c elasticsearch.cluster.name=docker-cluster 
Native controller process has stopped - no new native processes can be started | @timestamp=2024-10-14T07:30:21.651Z log.level=INFO ecs.version=1.2.0 service.name=ES_ECS event.dataset=elasticsearch.server process.thread.name=ml-cpp-log-tail-thread log.logger=org.elasticsearch.xpack.ml.process.NativeController elasticsearch.node.name=75fc863d6b9c elasticsearch.cluster.name=docker-cluster 
ERROR: Elasticsearch died while starting up, with exit code 78

If I change it to,

services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.15.2
    environment:
      xpack.security.http.ssl.enabled: false
      discovery.type: single-node

Problem solved. But this is only for single node cluster. I want to turn off TLS at HTTP layer on multi-node elasticsearch cluster. That is to expand through commands similar to the following. Setting discovery.type: single-node will cause /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token to fail to work.

docker compose up -d
docker compose exec es01 /bin/bash
/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic
y
/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana
exit

Or did I misunderstand the documentation? Can't I turn off TLS at the HTTP layer for a multi-node Elasticsearch cluster? Simply adding xpack.security.transport.ssl.enabled: true or discovery.seed_hosts: "[]" to docker-compose.yml does not allow Elasticsearch to start normally.

DavidTurner · October 14, 2024, 11:29am

There's some confusion here. TLS is required on the transport interface ...

... but not on the HTTP interface. If you are getting the message saying Transport SSL must be enabled then that must be because xpack.security.transport.ssl.enabled is set to false. Set this to true instead. If you want to disable TLS on the HTTP interface then set xpack.security.http.ssl.enabled to false instead.

This mode is only really for testing, you should not use it in production. See these docs for more info.

linghengqian · October 14, 2024, 12:09pm

@DavidTurner I switched to the following configuration,

services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.15.2
    environment:
      xpack.security.http.ssl.enabled: false
      xpack.security.transport.ssl.enabled: true

Now I can see that elasticsearch starts asking for xpack.security.transport.ssl.keystore.path.

fatal exception while booting Elasticsearch | @timestamp=2024-10-14T11:51:18.165Z log.level=ERROR ecs.version=1.2.0 service.name=ES_ECS event.dataset=elasticsearch.server process.thread.name=main log.logger=org.elasticsearch.bootstrap.Elasticsearch elasticsearch.node.name=a57097bb2ca6 elasticsearch.cluster.name=docker-cluster error.type=org.elasticsearch.ElasticsearchSecurityException error.message=invalid SSL configuration for xpack.security.transport.ssl - server ssl configuration requires a key and certificate, but these have not been configured; you must set either [xpack.security.transport.ssl.keystore.path], or both [xpack.security.transport.ssl.key] and [xpack.security.transport.ssl.certificate] error.stack_trace=org.elasticsearch.ElasticsearchSecurityException: invalid SSL configuration for xpack.security.transport.ssl - server ssl configuration requires a key and certificate, but these have not been configured; you must set either [xpack.security.transport.ssl.keystore.path], or both [xpack.security.transport.ssl.key] and [xpack.security.transport.ssl.certificate]
	at org.elasticsearch.xcore@8.15.2/org.elasticsearch.xpack.core.ssl.SSLService.throwExceptionForMissingKeyMaterial(SSLService.java:686)
	at org.elasticsearch.xcore@8.15.2/org.elasticsearch.xpack.core.ssl.SSLService.validateServerConfiguration(SSLService.java:642)
	at org.elasticsearch.xcore@8.15.2/org.elasticsearch.xpack.core.ssl.SSLService.loadSslConfigurations(SSLService.java:627)
	at org.elasticsearch.xcore@8.15.2/org.elasticsearch.xpack.core.ssl.SSLService.<init>(SSLService.java:160)
	at org.elasticsearch.xcore@8.15.2/org.elasticsearch.xpack.core.XPackPlugin.createSSLService(XPackPlugin.java:496)
	at org.elasticsearch.xcore@8.15.2/org.elasticsearch.xpack.core.XPackPlugin.createComponents(XPackPlugin.java:325)
	at org.elasticsearch.server@8.15.2/org.elasticsearch.node.NodeConstruction.lambda$construct$13(NodeConstruction.java:868)
	at org.elasticsearch.server@8.15.2/org.elasticsearch.plugins.PluginsService.lambda$flatMap$1(PluginsService.java:253)
	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:288)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
	at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:722)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:556)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:546)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:622)
	at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:291)
	at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:631)
	at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:637)
	at java.base/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:642)
	at org.elasticsearch.server@8.15.2/org.elasticsearch.node.NodeConstruction.construct(NodeConstruction.java:868)
	at org.elasticsearch.server@8.15.2/org.elasticsearch.node.NodeConstruction.prepareConstruction(NodeConstruction.java:270)
	at org.elasticsearch.server@8.15.2/org.elasticsearch.node.Node.<init>(Node.java:192)
	at org.elasticsearch.server@8.15.2/org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:242)
	at org.elasticsearch.server@8.15.2/org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:242)
	at org.elasticsearch.server@8.15.2/org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:76)
 
ERROR: Elasticsearch did not exit normally - check the logs at /usr/share/elasticsearch/logs/docker-cluster.log
ERROR: Elasticsearch died while starting up, with exit code 1

Further, I just want to know if there is an environment variable for elasticsearch that can enable the auto-configuration of the xpack.security.transport.ssl.** part alone to avoid manually generating certificate files? If there is no such environment variable, I think the current discussion can be closed, because it does seem more complicated than a step like xpack.security.autoconfiguration.enabled.

DavidTurner · October 14, 2024, 12:30pm

Perhaps, sorry, I'm not sure exactly which possibilities are or are not covered with auto-config and auto-enrollment. Still, manually generating certificates is not all that hard, and there's a step-by-step guide in the docs.