# Logstash fails after upgrading to version - 8.15.1 **URL:** \ **Category:** Logstash\ **Tags:** elastic-stack-security, docker\ **Created:** [September 10, 2024, 3:24pm UTC](https://discuss.elastic.co/t/logstash-fails-after-upgrading-to-version-8-15-1/366333 "2024-09-10T15:24:40Z")\ **Posts on this page:** 19\ **Page:** 1
**Author:** ![mohanss08](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohanss08/32/82297_2.png) [@mohanss08](https://discuss.elastic.co/u/mohanss08)\ **Post date:** [September 10, 2024, 3:24pm UTC](https://discuss.elastic.co/t/logstash-fails-after-upgrading-to-version-8-15-1/366333/1 "2024-09-10T15:24:40Z")
Hi Team, Today i upgraded my Elasticsearch, Logstash, and Kibana stack from version 8.6.2 to 8.15.1. My setup is based on Docker Compose. After the upgrade, the Logstash container fails to start and throws the following error: ```auto ERROR: Failed to load settings file from "path.settings". Aborting... path.settings=/usr/share/logstash/config, exception=LogStash::ConfigurationError, message=>Cannot evaluate ${xpack.monitoring.elasticsearch.password}. Replacement variable xpack.monitoring.elasticsearch.password is not defined in a Logstash secret store or as an Environment entry and there is no default value given. [FATAL] 2024-09-10 10:43:39.349 [main] Logstash - Logstash stopped processing because of an error: (SystemExit) exit ``` It seems that Logstash cannot find or evaluate the variable `${xpack.monitoring.elasticsearch.password}` which was working fine before the upgrade. I鈥檝e checked my environment variables and secret store, but I鈥檓 not sure what could be causing this issue after the version change. Could anyone advise on what might be causing this error or how to resolve it? Thank you in advance! ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [September 10, 2024, 3:54pm UTC](https://discuss.elastic.co/t/logstash-fails-after-upgrading-to-version-8-15-1/366333/2 "2024-09-10T15:54:49Z")
> [@mohanss08](#): > > Recently upgraded my Elasticsearch, Logstash, and Kibana stack from version 8.6.2 to 8.15.2 Do you mean 8.15.1? I do not see any indication 8.15.2 is out yet. 8.15.1 includes a [tweak](https://github.com/elastic/logstash/pull/16375) to the way ${} is evaluated and it looks like there is already an [open issue](https://github.com/elastic/logstash/issues/16437) for it. Does downgrading to 8.15.0 fix things? ---
**Author:** ![mohanss08](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohanss08/32/82297_2.png) [@mohanss08](https://discuss.elastic.co/u/mohanss08)\ **Post date:** [September 10, 2024, 4:04pm UTC](https://discuss.elastic.co/t/logstash-fails-after-upgrading-to-version-8-15-1/366333/3 "2024-09-10T16:04:48Z")
Hi @Badger - Sorry for the inconvenience!! yes it is `8.15.1` version. I haven't downgraded yet, So i will check the same and get back. ---
**Author:** ![mohanss08](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohanss08/32/82297_2.png) [@mohanss08](https://discuss.elastic.co/u/mohanss08)\ **Post date:** [September 11, 2024, 5:22am UTC](https://discuss.elastic.co/t/logstash-fails-after-upgrading-to-version-8-15-1/366333/4 "2024-09-11T05:22:07Z")
@Badger - I have downgraded my `Elasticsearch, Logstash, and Kibana stack version to 8.15.0`, but I'm still getting the same error in my Logstash container, The logs as follows. ```auto logstash | ERROR: Failed to load settings file from "path.settings". Aborting... path.setting=/usr/share/logstash/config, exception=LogStash::ConfigurationError, message=>Cannot evaluate `${xpack.monitoring.elasticsearch.password}`. Replacement variable `xpack.monitoring.elasticsearch.password` is not defined in a Logstash secret store or as an Environment entry and there is no default value given. logstash | [FATAL] 2024-09-11 05:12:22.470 [main] Logstash - Logstash stopped processing because of an error: (SystemExit) exit logstash | org.jruby.exceptions.SystemExit: (SystemExit) exit logstash | at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:921) ~[jruby.jar:?] logstash | at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:880) ~[jruby.jar:?] logstash | at usr.share.logstash.lib.bootstrap.environment.
(/usr/share/logstash/lib/bootstrap/environment.rb:90) ~[?:?] logstash | 2024/09/11 05:12:35 Setting 'xpack.monitoring.elasticsearch.password' from environment. logstash | 2024/09/11 05:12:35 Setting 'xpack.monitoring.enabled' from environment. logstash | 2024/09/11 05:12:35 Setting 'xpack.monitoring.elasticsearch.username' from environment. logstash | Using bundled JDK: /usr/share/logstash/jdk ``` Whereas in Elasticsearch, it says `cannot downgrade a node from version [8.15.1] to version [8.15.0]`. Here is the complete error message. ```auto elasticsearch | {"@timestamp":"2024-09-11T05:12:41.267Z", "log.level":"ERROR", "message":"fatal exception while booting Elasticsearch", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.bootstrap.Elasticsearch","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster","error.type":"java.lang.IllegalStateException","error.message":"cannot downgrade a node from version [8.15.1] to version [8.15.0]","error.stack_trace":"java.lang.IllegalStateException: cannot downgrade a node from version [8.15.1] to version [8.15.0]\n\tat org.elasticsearch.server@8.15.0/org.elasticsearch.env.NodeMetadata.verifyUpgradeToCurrentVersion(NodeMetadata.java:134)\n\tat org.elasticsearch.server@8.15.0/org.elasticsearch.env.NodeMetadata.upgradeToCurrentVersion(NodeMetadata.java:140)\n\tat org.elasticsearch.server@8.15.0/org.elasticsearch.env.NodeEnvironment.loadNodeMetadata(NodeEnvironment.java:637)\n\tat org.elasticsearch.server@8.15.0/org.elasticsearch.env.NodeEnvironment.(NodeEnvironment.java:334)\n\tat org.elasticsearch.server@8.15.0/org.elasticsearch.node.NodeConstruction.validateSettings(NodeConstruction.java:513)\n\tat org.elasticsearch.server@8.15.0/org.elasticsearch.node.NodeConstruction.prepareConstruction(NodeConstruction.java:260)\n\tat org.elasticsearch.server@8.15.0/org.elasticsearch.node.Node.(Node.java:192)\n\tat org.elasticsearch.server@8.15.0/org.elasticsearch.bootstrap.Elasticsearch$2.(Elasticsearch.java:242)\n\tat org.elasticsearch.server@8.15.0/org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:242)\n\tat org.elasticsearch.server@8.15.0/org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:76)\n"} elasticsearch | ERROR: Elasticsearch did not exit normally - check the logs at /usr/share/elasticsearch/logs/docker-cluster.log elasticsearch | elasticsearch | ERROR: Elasticsearch died while starting up, with exit code 1 ``` ---
**Author:** ![ErGeek](https://avatars.discourse-cdn.com/v4/letter/e/e5b9ba/32.png) [@ErGeek](https://discuss.elastic.co/u/ErGeek)\ **Post date:** [September 18, 2024, 3:44pm UTC](https://discuss.elastic.co/t/logstash-fails-after-upgrading-to-version-8-15-1/366333/5 "2024-09-18T15:44:50Z")
Hi @mohanss08 , Recently , I also upgraded my logstash to 8.15.1 version and am facing the same error as you. "LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"Java::JavaLang::IllegalStateException", :message=\>"Unable to configure plugins: Cannot evaluate `${BOOTSTRAP_SERVERS}`. Replacement variable `BOOTSTRAP_SERVERS` is not defined in a Logstash secret store or an environment entry and there is no default value given."," Were you able to get any success on this issue? Regards Nalin ---
**Author:** ![mohanss08](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohanss08/32/82297_2.png) [@mohanss08](https://discuss.elastic.co/u/mohanss08)\ **Post date:** [September 18, 2024, 4:02pm UTC](https://discuss.elastic.co/t/logstash-fails-after-upgrading-to-version-8-15-1/366333/6 "2024-09-18T16:02:16Z")
Hi Nalin, The problem still i couldn't able to fix it. May be we need try with next version - `8.15.2` ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [September 18, 2024, 5:14pm UTC](https://discuss.elastic.co/t/logstash-fails-after-upgrading-to-version-8-15-1/366333/7 "2024-09-18T17:14:55Z")
> [@mohanss08](#): > > Hi Nalin, The problem still i couldn't able to fix it Have you downgraded Logstash? You cannot downgrade Elasticsearch and Kibana, but you can downgrade Logstash, try to downgrade it to 8.14.3. ---
**Author:** ![mohanss08](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohanss08/32/82297_2.png) [@mohanss08](https://discuss.elastic.co/u/mohanss08)\ **Post date:** [September 19, 2024, 3:35am UTC](https://discuss.elastic.co/t/logstash-fails-after-upgrading-to-version-8-15-1/366333/8 "2024-09-19T03:35:31Z")
@leandrojmp Same problem occurring even with `logstash v-8.14.3` & `8.13.4` as well. ```auto logstash | ERROR: Failed to load settings file from "path.settings". Aborting... path.setting=/usr/share/logstash/config, exception=LogStash::ConfigurationError, message=>Cannot evaluate `${xpack.monitoring.elasticsearch.password}`. Replacement variable `xpack.monitoring.elasticsearch.password` is not defined in a Logstash secret store or as an Environment entry and there is no default value given. logstash | [FATAL] 2024-09-19 03:30:03.596 [main] Logstash - Logstash stopped processing because of an error: (SystemExit) exit logstash | org.jruby.exceptions.SystemExit: (SystemExit) exit logstash | at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:921) ~[jruby.jar:?] logstash | at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:880) ~[jruby.jar:?] logstash | at usr.share.logstash.lib.bootstrap.environment.
(/usr/share/logstash/lib/bootstrap/environment.rb:90) ~[?:?] logstash | 2024/09/19 03:30:04 Setting 'xpack.monitoring.elasticsearch.password' from environment. logstash | 2024/09/19 03:30:04 Setting 'xpack.monitoring.enabled' from environment. logstash | 2024/09/19 03:30:04 Setting 'xpack.monitoring.elasticsearch.username' from environment. logstash | Using bundled JDK: /usr/share/logstash/jdk ``` ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [September 19, 2024, 4:21am UTC](https://discuss.elastic.co/t/logstash-fails-after-upgrading-to-version-8-15-1/366333/9 "2024-09-19T04:21:25Z")
Yeah, just checked the issue linked, the bug was introduced on `8.13.1` as you can check [here](https://github.com/elastic/logstash/pull/16365). > After 8.13.1, running Logstash on docker with list environment variable (example: `-e XPACK_MANAGEMENT_ELASTICSEARCH_HOSTS='[es.host.1, es.host.2:9500]'` ) doesn't work. So you need to try with a version before `8.13.1` like `8.13.0` or `8.12.2`, you can also still use Logstash `8.6.2` where it was working. ---
**Author:** ![mohanss08](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohanss08/32/82297_2.png) [@mohanss08](https://discuss.elastic.co/u/mohanss08)\ **Post date:** [September 19, 2024, 6:42am UTC](https://discuss.elastic.co/t/logstash-fails-after-upgrading-to-version-8-15-1/366333/10 "2024-09-19T06:42:18Z")
@leandrojmp - Thanks it works with `logstash-8.12.2` version. ---
**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Post date:** [September 19, 2024, 8:16pm UTC](https://discuss.elastic.co/t/logstash-fails-after-upgrading-to-version-8-15-1/366333/11 "2024-09-19T20:16:48Z")
> [@leandrojmp](#): > > on `8.13.` Thanks for this topic. I was about to upgrade to 8.15. I will wait for new version ---
**Author:** ![mohanss08](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohanss08/32/82297_2.png) [@mohanss08](https://discuss.elastic.co/u/mohanss08)\ **Post date:** [October 9, 2024, 5:34am UTC](https://discuss.elastic.co/t/logstash-fails-after-upgrading-to-version-8-15-1/366333/12 "2024-10-09T05:34:05Z")
Hi All, Today tried with `Logstash - 8.15.2` version, however still same problem. ```auto logstash | ERROR: Failed to load settings file from "path.settings". Aborting... path.setting=/usr/share/logstash/config, exception=LogStash::ConfigurationError, message=>Cannot evaluate `${xpack.monitoring.elasticsearch.password}`. Replacement variable `xpack.monitoring.elasticsearch.password` is not defined in a Logstash secret store or as an Environment entry and there is no default value given. logstash | [FATAL] 2024-10-09 05:30:54.028 [main] Logstash - Logstash stopped processing because of an error: (SystemExit) exit logstash | org.jruby.exceptions.SystemExit: (SystemExit) exit logstash | at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:921) ~[jruby.jar:?] logstash | at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:880) ~[jruby.jar:?] logstash | at usr.share.logstash.lib.bootstrap.environment.
(/usr/share/logstash/lib/bootstrap/environment.rb:90) ~[?:?] logstash | 2024/10/09 05:30:54 Setting 'xpack.monitoring.elasticsearch.password' from environment. logstash | 2024/10/09 05:30:54 Setting 'xpack.monitoring.enabled' from environment. logstash | 2024/10/09 05:30:54 Setting 'xpack.monitoring.elasticsearch.username' from environment. logstash | Using bundled JDK: /usr/share/logstash/jdk ``` ---
**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Post date:** [October 9, 2024, 1:15pm UTC](https://discuss.elastic.co/t/logstash-fails-after-upgrading-to-version-8-15-1/366333/13 "2024-10-09T13:15:46Z")
Thanks for update. this means will wait for next version. 馃檪 ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [October 9, 2024, 1:34pm UTC](https://discuss.elastic.co/t/logstash-fails-after-upgrading-to-version-8-15-1/366333/14 "2024-10-09T13:34:07Z")
> [@mohanss08](#): > > Today tried with `Logstash - 8.15.2` version, however still same problem. This is marked as fixed on 8.15.2, so I would comment on the Github PR that marked this is fixed. It is this one: [Fixes the issue where LS wipes out all quotes from docker env variables. by mashhurs 路 Pull Request #16456 路 elastic/logstash 路 GitHub](https://github.com/elastic/logstash/pull/16456) ---
**Author:** ![mohanss08](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohanss08/32/82297_2.png) [@mohanss08](https://discuss.elastic.co/u/mohanss08)\ **Post date:** [October 9, 2024, 1:47pm UTC](https://discuss.elastic.co/t/logstash-fails-after-upgrading-to-version-8-15-1/366333/15 "2024-10-09T13:47:56Z")
@leandrojmp - Yes in the release notes page - [Logstash 8.15.2 Release Notes | Logstash Reference [8.15] | Elastic](https://www.elastic.co/guide/en/logstash/current/logstash-8-15-2.html) i could see that it is fixed, but i tried this morning it didn't work as expected like old version - `8.12.2`. ---
**Author:** ![mohanss08](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohanss08/32/82297_2.png) [@mohanss08](https://discuss.elastic.co/u/mohanss08)\ **Post date:** [October 10, 2024, 3:19am UTC](https://discuss.elastic.co/t/logstash-fails-after-upgrading-to-version-8-15-1/366333/16 "2024-10-10T03:19:21Z")
@leandrojmp - Is it something that im missing with `Logstash - 8.15.2`? ---
**Author:** ![mohanss08](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohanss08/32/82297_2.png) [@mohanss08](https://discuss.elastic.co/u/mohanss08)\ **Post date:** [October 16, 2024, 5:00am UTC](https://discuss.elastic.co/t/logstash-fails-after-upgrading-to-version-8-15-1/366333/17 "2024-10-16T05:00:50Z")
Hi All, Since Logstash is functioning properly with `v8.12.2` even though it is stated that it has been resolved with `v8.15.2`, But it is not working correctly as expected. So, please let me know whether I should wait for the next release? ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [October 16, 2024, 11:52am UTC](https://discuss.elastic.co/t/logstash-fails-after-upgrading-to-version-8-15-1/366333/18 "2024-10-16T11:52:47Z")
> [@mohanss08](#): > > So, please let me know whether I should wait for the next release? Did you report it again to Elastic by commenting on the closing issue or opening a new issue? If you do not report it as a bug or unsolved, Elastic will not look into it. You need to open a new issue or comment on this [one](https://github.com/elastic/logstash/pull/16456) with evidences that the fix didn't work. ---
**Author:** ![mohanss08](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohanss08/32/82297_2.png) [@mohanss08](https://discuss.elastic.co/u/mohanss08)\ **Post date:** [October 18, 2024, 4:35am UTC](https://discuss.elastic.co/t/logstash-fails-after-upgrading-to-version-8-15-1/366333/19 "2024-10-18T04:35:07Z")
Ok thanks @leandrojmp - Issue has been registered here [[Logstash container fails to start 路 Issue #16563 路 elastic/logstash 路 GitHub](https://github.com/elastic/logstash/issues/16563)]