I added X_PWD
What's strange is when I run the list command I get an error
/usr/share/logstash/bin/logstash-keystore --path.settings /etc/logstash list
[ERROR] 2019-08-25 13:27:20.827 [main] secretstorecli - Found a file at /etc/logstash/logstash.keystore, but it is not a valid Logstash keystore. {:cause=>java.io.IOException: Integrity check failed: java.io.IOException: getSecretKey failed: Password is not ASCII, :backtrace=>["org.logstash.secret.store.backend.JavaKeyStore.load(org/logstash/secret/store/backend/JavaKeyStore.java:268)", "org.logstash.secret.store.backend.JavaKeyStore.load(org/logstash/secret/store/backend/JavaKeyStore.java:40)", "org.logstash.secret.store.SecretStoreFactory.doIt(org/logstash/secret/store/SecretStoreFactory.java:109)", "org.logstash.secret.store.SecretStoreFactory.load(org/logstash/secret/store/SecretStoreFactory.java:95)", "org.logstash.secret.cli.SecretStoreCli.command(org/logstash/secret/cli/SecretStoreCli.java:77)", "java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)", "org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:425)", "org.jruby.javasupport.JavaMethod.invokeDirect(org/jruby/javasupport/JavaMethod.java:292)", "RUBY.<class:SecretStoreCli>(/usr/share/logstash/lib/secretstore/cli.rb:35)", "RUBY.<main>(/usr/share/logstash/lib/secretstore/cli.rb:16)", "org.jruby.Ruby.runInterpreter(org/jruby/Ruby.java:891)", "org.jruby.Ruby.runInterpreter(org/jruby/Ruby.java:895)", "org.jruby.Ruby.runNormally(org/jruby/Ruby.java:784)", "org.jruby.Ruby.runNormally(org/jruby/Ruby.java:797)", "org.jruby.Ruby.runFromMain(org/jruby/Ruby.java:609)", "org.jruby.Main.doRunFromMain(org/jruby/Main.java:415)", "org.jruby.Main.internalRun(org/jruby/Main.java:307)", "org.jruby.Main.run(org/jruby/Main.java:234)", "org.jruby.Main.main(org/jruby/Main.java:206)"]}
So I recreated everything
set +o history
export LOGSTASH_KEYSTORE_PASS=mypassword
set -o history
/usr/share/logstash/bin/logstash-keystore --path.settings /etc/logstash add es_pwd
2019-08-25 13:31:30,031 main ERROR Unable to locate appender "${sys:ls.log.format}_console" for logger config "root"
2019-08-25 13:31:30,031 main ERROR Unable to locate appender "${sys:ls.log.format}_rolling" for logger config "root"
2019-08-25 13:31:30,032 main ERROR Unable to locate appender "${sys:ls.log.format}_rolling_slowlog" for logger config "slowlog"
2019-08-25 13:31:30,032 main ERROR Unable to locate appender "${sys:ls.log.format}_console_slowlog" for logger config "slowlog"
Enter value for es_pwd:
Added 'es_pwd' to the Logstash keystore.
In logstash I set
xpack.monitoring.elasticsearch.password: $es_pwd
tail -F /var/log/logstash/logstash-plain.log
[2019-08-25T13:33:12,700][INFO ][logstash.javapipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, :thread=>"#<Thread:0x588aa324 run>"}
[2019-08-25T13:33:13,820][INFO ][logstash.inputs.beats ] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2019-08-25T13:33:13,875][INFO ][logstash.javapipeline ] Pipeline started {"pipeline.id"=>"main"}
[2019-08-25T13:33:14,006][INFO ][logstash.inputs.udp ] Starting UDP listener {:address=>"0.0.0.0:5045"}
[2019-08-25T13:33:14,032][INFO ][logstash.inputs.udp ] Starting UDP listener {:address=>"0.0.0.0:5047"}
[2019-08-25T13:33:14,190][INFO ][org.logstash.beats.Server] Starting server on port: 5044
[2019-08-25T13:33:14,222][INFO ][logstash.inputs.udp ] UDP listener started {:address=>"0.0.0.0:5047", :receive_buffer_bytes=>"106496", :queue_size=>"2000"}
[2019-08-25T13:33:14,232][INFO ][logstash.inputs.udp ] UDP listener started {:address=>"0.0.0.0:5045", :receive_buffer_bytes=>"106496", :queue_size=>"2000"}
[2019-08-25T13:33:14,269][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2019-08-25T13:33:14,727][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2019-08-25T13:33:37,297][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"Got response code '401' contacting Elasticsearch at URL 'https://localhost:9200/_xpack'"}
[2019-08-25T13:34:07,221][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"Got response code '401' contacting Elasticsearch at URL 'https://localhost:9200/_xpack'"}
I also tried adding the x_pwd to the keystore and setting
xpack.monitoring.elasticsearch.password: $x_pwd
Same thing
Thanks for your help