Logstash keystore value not detected in configuration processing

Hello Everyone,

I'm trying keystore to store the password for elasticsearch but for some reason I've encountered this error during configuration test.

# /usr/share/logstash/bin/logstash --path.settings /etc/logstash -t
Using bundled JDK: /usr/share/logstash/jdk
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bundler-1.17.3/lib/bundler/rubygems_integration.rb:200: warning: constant Gem::ConfigMap is deprecated
ERROR: Failed to load settings file from "path.settings". Aborting... path.setting=/etc/logstash, exception=LogStash::ConfigurationError, message=>Cannot evaluate `${es_password}`. Replacement variable `es_password` is not defined in a Logstash secret store or as an Environment entry and there is no default value given.
[FATAL] 2021-08-19 05:35:33.386 [main] Logstash - Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
	at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.19.0.jar:?]
	at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.19.0.jar:?]
	at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:89) ~[?:?]

Keystore data does exist:

# bin/logstash-keystore list
es_password

And here's the basic config of my logstash.yml file:

# cat logstash.yml
path.data: /var/lib/logstash
path.logs: /var/log/logstash

xpack.monitoring.elasticsearch.password: ${es_password}
xpack.monitoring.elasticsearch.url: https://localhost:9200
xpack.monitoring.elasticsearch.username: logstash_system

Please let me know how can I make it work. Any advise is highly appreciated.

Did anyone encounter similar issue?

Hi @ramilbermejo welcome to the community.

Most likely Logstash can't find the keystore.

What version?

How was it installed?

See here

Did you use path.settings when you created it?

The keystore must be located in the Logstash path.settings directory. This is the same directory that contains the logstash.yml file. When performing any operation against the keystore, it is recommended to set path.settings for the keystore command.

set +o history
export LOGSTASH_KEYSTORE_PASS=mypassword
set -o history
sudo -E /usr/share/logstash/bin/logstash-keystore --path.settings /etc/logstash create

@stephenb thanks for the reply.

My test environment has logstash version 7.14.0.

I've followed the same guide when I set it up but I might missed something so I'll further check and review.

Thanks again!

Another possibility, once you create the keystore and make sure it is in the right location make sure the user you are starting Logstash with has permission to access the keystore

1 Like

OK, thank you!