Fail to generate certs

Elastic Stack Elasticsearch
1

Hello I'm trying to deploy elasticsearch and kibana using docker-compose but the elasticsearch container exited because the file usr/share/elasticsearch/config/certs/transport.p12 does not exists.

Precision 1 : I want to connect with kibana using a wireguard VPN (that's why I use 10.0.0.1 and the "vpn" network) and I want kibana discuss with elasticsearch using the "internal" network (using docker container names)
Precision 2 : For the first deployment ELASTIC_PASSWORD and KIBANA_PASSWORD are empty, that's why I use ${ELASTIC_PASSWORD:-} syntax, I know I have to generate manually a new elastic and kibana_system password and set the env variables for further redeployment

Here is my docker-compose es/kibana part:

elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.17.4
    container_name: elasticsearch
    environment:
      - xpack.security.enabled=true
      - xpack.security.enrollment.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.keystore.path=certs/http.p12
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.security.transport.ssl.keystore.path=certs/transport.p12
      - xpack.security.transport.ssl.truststore.path=certs/transport.p12

      - bootstrap.memory_lock=true
      # - xpack.license.self_generated.type=trial
      - network.publish_host=elasticsearch
      - ES_JAVA_OPTS=-Xms2g -Xmx2g
      - ELASTIC_PASSWORD=${ELASTIC_PASSWORD:-}
      - ELASTIC_CONTAINER=true
      - discovery.type=single-node
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - ./elasticsearch/data:/usr/share/elasticsearch/data
      # - ./elasticsearch/config:/usr/share/elasticsearch/config
      - ./elasticsearch/plugins:/usr/share/elasticsearch/plugins
      - ./elasticsearch/logs:/usr/share/elasticsearch/logs

    user: "1000:1000"
    networks:
      - internal
      - vpn

    ports:
      - "10.0.0.1:9200:9200"
      - "127.0.0.1:9200:9200"

    deploy:
      resources:
        limits:
          memory: 4G 

  kibana:
    image: docker.elastic.co/kibana/kibana:8.17.4
    container_name: kibana
    depends_on:
      - elasticsearch
    environment:
      - SERVER_PUBLICBASEURL=https://elasticsearch:9200
      - ELASTICSEARCH_HOSTS=https://elasticsearch:9200
      - ELASTICSEARCH_SSL_VERIFICATION_MODE=full
      - ELASTICSEARCH_USERNAME=kibana_system
      - ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD:-}
      - ELASTICSEARCH_SSL_CERTIFICATE_AUTHORITIES=/usr/share/kibana/config/certs/http_ca.crt
      
    volumes:
      - ./elasticsearch/certificates:/usr/share/kibana/config/certs

    ports:
      - "10.0.0.1:5601:5601"
    networks:
      - internal
      - vpn

here are the logs of the elasticsearch container:

fatal exception while booting Elasticsearch | @timestamp=2025-04-01T09:09:21.211Z log.level=ERROR ecs.version=1.2.0 service.name=ES_ECS event.dataset=elasticsearch.server process.thread.name=main log.logger=org.elasticsearch.bootstrap.Elasticsearch elasticsearch.node.name=156576dd69e5 elasticsearch.cluster.name=docker-cluster error.type=org.elasticsearch.ElasticsearchSecurityException error.message=failed to load SSL configuration [xpack.security.transport.ssl] - cannot read configured [PKCS12] keystore (as a truststore) [/usr/share/elasticsearch/config/certs/transport.p12] because the file does not exist error.stack_trace=org.elasticsearch.ElasticsearchSecurityException: failed to load SSL configuration [xpack.security.transport.ssl] - cannot read configured [PKCS12] keystore (as a truststore) [/usr/share/elasticsearch/config/certs/transport.p12] because the file does not exist
at org.elasticsearch.xcore@8.17.4/org.elasticsearch.xpack.core.ssl.SSLService.lambda$loadSslConfigurations$11(SSLService.java:620)
at java.base/java.util.HashMap.forEach(HashMap.java:1430)
at java.base/java.util.Collections$UnmodifiableMap.forEach(Collections.java:1708)
at org.elasticsearch.xcore@8.17.4/org.elasticsearch.xpack.core.ssl.SSLService.loadSslConfigurations(SSLService.java:616)
at org.elasticsearch.xcore@8.17.4/org.elasticsearch.xpack.core.ssl.SSLService.(SSLService.java:160)
at org.elasticsearch.xcore@8.17.4/org.elasticsearch.xpack.core.XPackPlugin.createSSLService(XPackPlugin.java:496)
at org.elasticsearch.xcore@8.17.4/org.elasticsearch.xpack.core.XPackPlugin.createComponents(XPackPlugin.java:325)
at org.elasticsearch.server@8.17.4/org.elasticsearch.node.NodeConstruction.lambda$construct$16(NodeConstruction.java:894)
at org.elasticsearch.server@8.17.4/org.elasticsearch.plugins.PluginsService.lambda$flatMap$1(PluginsService.java:254)
at java.base/java.util.stream.ReferencePipeline$7$1FlatMap.accept(ReferencePipeline.java:289)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:722)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:570)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:560)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:636)
at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:291)
at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:656)
at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:662)
at java.base/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:667)
at org.elasticsearch.server@8.17.4/org.elasticsearch.node.NodeConstruction.construct(NodeConstruction.java:916)
at org.elasticsearch.server@8.17.4/org.elasticsearch.node.NodeConstruction.prepareConstruction(NodeConstruction.java:291)
at org.elasticsearch.server@8.17.4/org.elasticsearch.node.Node.(Node.java:200)
at org.elasticsearch.server@8.17.4/org.elasticsearch.bootstrap.Elasticsearch$2.(Elasticsearch.java:247)
at org.elasticsearch.server@8.17.4/org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:247)
at org.elasticsearch.server@8.17.4/org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:76)
Caused by: org.elasticsearch.common.ssl.SslConfigException: cannot read configured [PKCS12] keystore (as a truststore) [/usr/share/elasticsearch/config/certs/transport.p12] because the file does not exist
at org.elasticsearch.sslconfig@8.17.4/org.elasticsearch.common.ssl.SslFileUtil.fileNotFound(SslFileUtil.java:67)
at org.elasticsearch.sslconfig@8.17.4/org.elasticsearch.common.ssl.SslFileUtil.ioException(SslFileUtil.java:38)
at org.elasticsearch.sslconfig@8.17.4/org.elasticsearch.common.ssl.StoreTrustConfig.readKeyStore(StoreTrustConfig.java:99)
at org.elasticsearch.sslconfig@8.17.4/org.elasticsearch.common.ssl.StoreTrustConfig.createTrustManager(StoreTrustConfig.java:83)
at org.elasticsearch.xcore@8.17.4/org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:479)
at java.base/java.util.HashMap.computeIfAbsent(HashMap.java:1229)
at org.elasticsearch.xcore@8.17.4/org.elasticsearch.xpack.core.ssl.SSLService.lambda$loadSslConfigurations$11(SSLService.java:618)
... 24 more
Caused by: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/config/certs/transport.p12
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:262)
at java.base/java.nio.file.Files.newByteChannel(Files.java:380)
at java.base/java.nio.file.Files.newByteChannel(Files.java:432)
at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420)
at java.base/java.nio.file.Files.newInputStream(Files.java:160)
at org.elasticsearch.sslconfig@8.17.4/org.elasticsearch.common.ssl.KeyStoreUtil.readKeyStore(KeyStoreUtil.java:72)
at org.elasticsearch.sslconfig@8.17.4/org.elasticsearch.common.ssl.StoreTrustConfig.readKeyStore(StoreTrustConfig.java:95)

2

Yes, that's right.

You have this setting:

      - xpack.security.enabled=true

That automatically sets the node into "manual configuration mode" and it will not do any automatic security configuration or generate any certificates.

But this

      - xpack.security.transport.ssl.keystore.path=certs/transport.p12

Requires that the file exist. Since you don't provide, and you've configured the node not to generate certs, the file doesn't exist, and it fails.

But..

      - discovery.type=single-node

Since you're only running a single node, you don't need any transport.ssl settings.

3

Ok perhaps, but so why the elasticsearch.yml base configuration file in the official docker image is this one :

cluster.name: "docker-cluster"
network.host: 0.0.0.0

#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically      
# generated to configure Elasticsearch security features on 31-03-2025 07:30:12
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
cluster.initial_master_nodes: ["08ad56e9a6e0"]

#----------------------- END SECURITY AUTO CONFIGURATION -------------------------

Basically if I run (without docker-compose, directly in a terminal) :
docker run --name es01 --net elastic -p 9200:9200 -it -m 1GB docker.elastic.co/elasticsearch/elasticsearch:8.17.4

It install es and generate a certs directory with certificates files in the config directory, yet the default elasticsearch.yml file has the xpack.security.enabled: true ....