# Fatal exception while booting Elasticsearch

**URL:** <https://discuss.elastic.co/t/fatal-exception-while-booting-elasticsearch/330887>\
**Category:** Elasticsearch\
**Tags:** painless, runtime-fields\
**Created:** [April 26, 2023, 10:12pm UTC](https://discuss.elastic.co/t/fatal-exception-while-booting-elasticsearch/330887 "2023-04-26T22:12:33Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Lelc79](https://avatars.discourse-cdn.com/v4/letter/l/43a26b/32.png) [@Lelc79](https://discuss.elastic.co/u/Lelc79)\
**Post date:** [April 26, 2023, 10:12pm UTC](https://discuss.elastic.co/t/fatal-exception-while-booting-elasticsearch/330887/1 "2023-04-26T22:12:33Z")

</div>

Hi community

My elasticsearch was Up until that i changed all the passwords.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/4/6460866e0d349fc140a343e36c91761195414638.jpeg)

After that i tried starting elasticsearch services and it didn't work.  
When i saw the logs in /var/log/elasticsearch/elasticsearch.log  
i get it:

```auto
[2023-04-26T17:00:57,543][INFO][o.e.n.Node] [DEMO] version[8.7.0], pid[45025], build[deb/09520b59b6bc1057340b55750186466ea715e30e/2023-03-27T16:31:09.816451435Z], OS[Linux/5.10.0-21-amd64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/19.0.2/19.0.2+7-44]
[2023-04-26T17:00:57,551][INFO][o.e.n.Node] [DEMO] JVM home [/usr/share/elasticsearch/jdk], using bundled JDK [true]
[2023-04-26T17:00:57,551][INFO][o.e.n.Node] [DEMO] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=ALL-UNNAMED, -Xms2g, -Xmx2g, -XX:+UseG1GC, -Djava.io.tmpdir=/tmp/elasticsearch-12732300502209698790, -XX:+HeapDumpOnOutOfMemoryError, -XX:+ExitOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=1073741824, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, -Des.distribution.type=deb, --module-path=/usr/share/elasticsearch/lib, --add-modules=jdk.net, -Djdk.module.main=org.elasticsearch.server]
[2023-04-26T17:01:00,491][INFO][c.a.c.i.j.JacksonVersion] [DEMO] Package versions: jackson-core=2.13.4, jackson-databind=2.13.4-2, jackson-dataformat-xml=2.13.4, jackson-datatype-jsr310=2.13.4, azure-core=1.34.0, Troubleshooting version conflicts: https://aka.ms/azsdk/java/dependency/troubleshoot
[2023-04-26T17:01:02,744][INFO][o.e.p.PluginsService] [DEMO] loaded module [aggregations]
[2023-04-26T17:01:02,744][INFO][o.e.p.PluginsService] [DEMO] loaded module [analysis-common]
[2023-04-26T17:01:02,744][INFO][o.e.p.PluginsService] [DEMO] loaded module [apm]
[2023-04-26T17:01:02,745][INFO][o.e.p.PluginsService] [DEMO] loaded module [blob-cache]
[2023-04-26T17:01:02,745][INFO][o.e.p.PluginsService] [DEMO] loaded module [constant-keyword]
[2023-04-26T17:01:02,745][INFO][o.e.p.PluginsService] [DEMO] loaded module [data-streams]
[2023-04-26T17:01:02,746][INFO][o.e.p.PluginsService] [DEMO] loaded module [frozen-indices]
[2023-04-26T17:01:02,746][INFO][o.e.p.PluginsService] [DEMO] loaded module [ingest-attachment]
[2023-04-26T17:01:02,746][INFO][o.e.p.PluginsService] [DEMO] loaded module [ingest-common]
[2023-04-26T17:01:02,747][INFO][o.e.p.PluginsService] [DEMO] loaded module [ingest-geoip]
[2023-04-26T17:01:02,747][INFO][o.e.p.PluginsService] [DEMO] loaded module [ingest-user-agent]
[2023-04-26T17:01:02,747][INFO][o.e.p.PluginsService] [DEMO] loaded module [kibana]
[2023-04-26T17:01:02,748][INFO][o.e.p.PluginsService] [DEMO] loaded module [lang-expression]
[2023-04-26T17:01:02,748][INFO][o.e.p.PluginsService] [DEMO] loaded module [lang-mustache]
[2023-04-26T17:01:02,748][INFO][o.e.p.PluginsService] [DEMO] loaded module [lang-painless]
[2023-04-26T17:01:02,749][INFO][o.e.p.PluginsService] [DEMO] loaded module [legacy-geo]
[2023-04-26T17:01:02,749][INFO][o.e.p.PluginsService] [DEMO] loaded module [mapper-extras]
[2023-04-26T17:01:02,749][INFO][o.e.p.PluginsService] [DEMO] loaded module [mapper-version]
[2023-04-26T17:01:02,749][INFO][o.e.p.PluginsService] [DEMO] loaded module [old-lucene-versions]
[2023-04-26T17:01:02,750][INFO][o.e.p.PluginsService] [DEMO] loaded module [parent-join]
[2023-04-26T17:01:02,750][INFO][o.e.p.PluginsService] [DEMO] loaded module [percolator]
[2023-04-26T17:01:02,750][INFO][o.e.p.PluginsService] [DEMO] loaded module [rank-eval]
[2023-04-26T17:01:02,751][INFO][o.e.p.PluginsService] [DEMO] loaded module [reindex]
[2023-04-26T17:01:02,751][INFO][o.e.p.PluginsService] [DEMO] loaded module [repositories-metering-api]
[2023-04-26T17:01:02,751][INFO][o.e.p.PluginsService] [DEMO] loaded module [repository-azure]
[2023-04-26T17:01:02,751][INFO][o.e.p.PluginsService] [DEMO] loaded module [repository-gcs]
[2023-04-26T17:01:02,752][INFO][o.e.p.PluginsService] [DEMO] loaded module [repository-s3]
[2023-04-26T17:01:02,752][INFO][o.e.p.PluginsService] [DEMO] loaded module [repository-url]
[2023-04-26T17:01:02,752][INFO][o.e.p.PluginsService] [DEMO] loaded module [runtime-fields-common]
[2023-04-26T17:01:02,753][INFO][o.e.p.PluginsService] [DEMO] loaded module [search-business-rules]
[2023-04-26T17:01:02,753][INFO][o.e.p.PluginsService] [DEMO] loaded module [searchable-snapshots]
[2023-04-26T17:01:02,754][INFO][o.e.p.PluginsService] [DEMO] loaded module [snapshot-based-recoveries]
[2023-04-26T17:01:02,754][INFO][o.e.p.PluginsService] [DEMO] loaded module [snapshot-repo-test-kit]
[2023-04-26T17:01:02,754][INFO][o.e.p.PluginsService] [DEMO] loaded module [spatial]
[2023-04-26T17:01:02,755][INFO][o.e.p.PluginsService] [DEMO] loaded module [systemd]
[2023-04-26T17:01:02,755][INFO][o.e.p.PluginsService] [DEMO] loaded module [transform]
[2023-04-26T17:01:02,755][INFO][o.e.p.PluginsService] [DEMO] loaded module [transport-netty4]
[2023-04-26T17:01:02,756][INFO][o.e.p.PluginsService] [DEMO] loaded module [unsigned-long]
[2023-04-26T17:01:02,756][INFO][o.e.p.PluginsService] [DEMO] loaded module [vector-tile]
[2023-04-26T17:01:02,756][INFO][o.e.p.PluginsService] [DEMO] loaded module [wildcard]
[2023-04-26T17:01:02,757][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-aggregate-metric]
[2023-04-26T17:01:02,757][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-analytics]
[2023-04-26T17:01:02,757][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-async]
[2023-04-26T17:01:02,758][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-async-search]
[2023-04-26T17:01:02,758][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-autoscaling]
[2023-04-26T17:01:02,759][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-ccr]
[2023-04-26T17:01:02,759][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-core]
[2023-04-26T17:01:02,759][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-deprecation]
[2023-04-26T17:01:02,760][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-enrich]
[2023-04-26T17:01:02,760][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-eql]
[2023-04-26T17:01:02,760][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-fleet]
[2023-04-26T17:01:02,760][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-graph]
[2023-04-26T17:01:02,761][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-identity-provider]
[2023-04-26T17:01:02,761][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-ilm]
[2023-04-26T17:01:02,761][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-logstash]
[2023-04-26T17:01:02,762][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-ml]
[2023-04-26T17:01:02,762][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-monitoring]
[2023-04-26T17:01:02,763][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-profiling]
[2023-04-26T17:01:02,763][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-ql]
[2023-04-26T17:01:02,763][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-rollup]
[2023-04-26T17:01:02,764][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-security]
[2023-04-26T17:01:02,764][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-shutdown]
[2023-04-26T17:01:02,764][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-sql]
[2023-04-26T17:01:02,764][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-stack]
[2023-04-26T17:01:02,765][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-text-structure]
[2023-04-26T17:01:02,765][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-voting-only-node]
[2023-04-26T17:01:02,765][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-watcher]
[2023-04-26T17:01:02,766][INFO][o.e.p.PluginsService] [DEMO] loaded module [x-pack-write-load-forecaster]
[2023-04-26T17:01:02,766][INFO][o.e.p.PluginsService] [DEMO] no plugins loaded
[2023-04-26T17:01:07,878][ERROR][o.e.b.Elasticsearch] [DEMO] fatal exception while booting Elasticsearch
java.lang.IllegalArgumentException: unknown secure setting [logstash] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
        at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:575) ~[elasticsearch-8.7.0.jar:?]
        at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:521) ~[elasticsearch-8.7.0.jar:?]
        at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:491) ~[elasticsearch-8.7.0.jar:?]
        at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:461) ~[elasticsearch-8.7.0.jar:?]
        at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:151) ~[elasticsearch-8.7.0.jar:?]
        at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:56) ~[elasticsearch-8.7.0.jar:?]
        at org.elasticsearch.node.Node.<init>(Node.java:474) ~[elasticsearch-8.7.0.jar:?]
        at org.elasticsearch.node.Node.<init>(Node.java:324) ~[elasticsearch-8.7.0.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:216) ~[elasticsearch-8.7.0.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:216) ~[elasticsearch-8.7.0.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:67) ~[elasticsearch-8.7.0.jar:?]
        Suppressed: java.lang.IllegalArgumentException: unknown secure setting [apm_system] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
                at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:575) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:521) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:491) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:461) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:151) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:56) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.node.Node.<init>(Node.java:474) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.node.Node.<init>(Node.java:324) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:216) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:216) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:67) ~[elasticsearch-8.7.0.jar:?]
        Suppressed: java.lang.IllegalArgumentException: unknown secure setting [kibana_system.password] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
                at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:575) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:521) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:491) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:461) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:151) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:56) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.node.Node.<init>(Node.java:474) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.node.Node.<init>(Node.java:324) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:216) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:216) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:67) ~[elasticsearch-8.7.0.jar:?]
        Suppressed: java.lang.IllegalArgumentException: unknown secure setting [beats_system] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
                at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:575) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:521) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:491) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:461) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:151) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:56) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.node.Node.<init>(Node.java:474) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.node.Node.<init>(Node.java:324) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:216) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:216) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:67) ~[elasticsearch-8.7.0.jar:?]
        Suppressed: java.lang.IllegalArgumentException: unknown secure setting [kibana_system] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
                at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:575) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:521) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:491) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:461) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:151) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:56) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.node.Node.<init>(Node.java:474) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.node.Node.<init>(Node.java:324) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:216) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:216) ~[elasticsearch-8.7.0.jar:?]
                at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:67) ~[elasticsearch-8.7.0.jar:?]

```

¿do you know how can i change the users passwords and enrolled with elasticsearch with theirs new passwords? or ¿is it other problem ?

thanks

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 27, 2023, 12:09am UTC](https://discuss.elastic.co/t/fatal-exception-while-booting-elasticsearch/330887/2 "2023-04-27T00:09:45Z")

</div>

The error message seems to say that you created a secure setting named logstash. You should just remove it as it's not a known setting.

---

<div class="post-metadata">

**Author:** ![Lelc79](https://avatars.discourse-cdn.com/v4/letter/l/43a26b/32.png) [@Lelc79](https://discuss.elastic.co/u/Lelc79)\
**Post date:** [April 27, 2023, 12:21am UTC](https://discuss.elastic.co/t/fatal-exception-while-booting-elasticsearch/330887/3 "2023-04-27T00:21:34Z")

</div>

Thanks Dadoonet

¿where do i remove that?  
As well as the logstash one, I also get the users of kibana, bootstrap

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 27, 2023, 3:41am UTC](https://discuss.elastic.co/t/fatal-exception-while-booting-elasticsearch/330887/4 "2023-04-27T03:41:39Z")

</div>

Run this:

```
bin/elasticsearch-keystore list

```

Then remove what is not needed. See [elasticsearch-keystore | Elasticsearch Guide [8.7] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/elasticsearch-keystore.html)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 25, 2023, 3:42am UTC](https://discuss.elastic.co/t/fatal-exception-while-booting-elasticsearch/330887/5 "2023-05-25T03:42:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.