Fatal exception while booting Elasticsearch | Missing secret key for s3 client

Elastic Stack Elasticsearch
1

I run it as a docker container in swarm environment. The error I got:

stamp":"2024-09-20T07:03:20.868Z", "log.level":"ERROR", "message":"fatal exception while booting Elasticsearch", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.bootstrap.Elasticsearch","elasticsearch.node.name":"es-1","elasticsearch.cluster.name":"docker-cluster","error.type":"java.lang.IllegalArgumentException","error.message":"Missing secret key for s3 client [default]","error.stack_trace":"java.lang.IllegalArgumentException: Missing secret key for s3 client [default]\n\tat org.elasticsearch.repositories.s3.S3ClientSettings.loadCredentials(S3ClientSettings.java:374)\n\tat org.elasticsearch.repositories.s3.S3ClientSettings.getClientSettings(S3ClientSettings.java:396)\n\tat org.elasticsearch.repositories.s3.S3ClientSettings.load(S3ClientSettings.java:315)\n\tat org.elasticsearch.repositories.s3.S3RepositoryPlugin.createComponents(S3RepositoryPlugin.java:108)\n\tat org.elasticsearch.server@8.6.0/org.elasticsearch.node.Node.lambda$new$16(Node.java:721)\n\tat org.elasticsearch.server@8.6.0/org.elasticsearch.plugins.PluginsService.lambda$flatMap$0(PluginsService.java:252)\n\tat java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273)\n\tat java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)\n\tat java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:722)\n\tat java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)\n\tat java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)\n\tat java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575)\n\tat java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)\n\tat java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:616)\n\tat java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:622)\n\tat java.base/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:627)\n\tat org.elasticsearch.server@8.6.0/org.elasticsearch.node.Node.<init>(Node.java:736)\n\tat org.elasticsearch.server@8.6.0/org.elasticsearch.node.Node.<init>(Node.java:322)\n\tat org.elasticsearch.server@8.6.0/org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:214)\n\tat org.elasticsearch.server@8.6.0/org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:214)\n\tat org.elasticsearch.server@8.6.0/org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:67)\n"}
ERROR: Elasticsearch exited unexpectedly
ERROR: Elasticsearch did not exit normally - check the logs at /usr/share/elasticsearch/logs/docker-cluster.log

If I can't start it then how can I set s3 secret key?

Edit:
I don't want to change elasticsearch.yml.
I guess this error is caused by following steps:

  1. I run bin/elasticsearch-keystore add -f s3.client.default.access_key in the container.
  2. The container somehow restarted, so I haven't had the chance to run bin/elasticsearch-keystore add -f s3.client.default.secret_key.

What can I do now?

2

Thanks for reaching out, @ACoder. I have a few follow up questions here.

  • What version of Elastic are you using?
  • Are you using docker secrets or environment variables for the s3 secret key ?