Logstash keystore encoding error

Elastic Stack Logstash
1

I try to set up a password protected Logstash keystore. It works when I run Logstash from the command line (as root, with set environment variable for the keystore), but when running as a systemd service (installed from deb package) it fails with:

Jul 31 14:00:06 my-server01 logstash[7380]: [2019-07-31T14:00:06,606][ERROR][logstash.config.sourceloader] Could not fetch all the sources {:exception=>LogStash::ConfigLoadingError, :message=>"The following config files contains non-ascii characters but are not UTF-8 encoded ["/etc/logstash/logstash.keystore"]", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/config/source/local.rb:83:in `read'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/local.rb:94:in `read'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/local.rb:190:in `local_pipeline_configs'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/local.rb:161:in `pipeline_configs'", "/usr/share/logstash/logstash-core/lib/logstash/config/source_loader.rb:61:in `block in fetch'", "org/jruby/RubyArray.java:2572:in `collect'", "/usr/share/logstash/logstash-core/lib/logstash/config/source_loader.rb:60:in `fetch'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:148:in `converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:96:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:367:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}

Jul 31 14:00:06 my-server01 logstash[7380]: [2019-07-31T14:00:06,615][ERROR][logstash.agent ] An exception happened when converging configuration {:exception=>RuntimeError, :message=>"Could not fetch the configuration, message: The following config files contains non-ascii characters but are not UTF-8 encoded ["/etc/logstash/logstash.keystore"]", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/agent.rb:155:in converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:96:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:367:in block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/task.rb:24:in block in initialize'"]}

I created the keystore this way:

  • Created file /etc/sysconfig/logstash, owner root, permissions 600, with content:
    LOGSTASH_KEYSTORE_PASS=mypassword
  • On terminal as root:
    set +o history
    export LOGSTASH_KEYSTORE_PASS=mypassword
    set -o history
    sudo -E /usr/share/logstash/bin/logstash-keystore --path.settings /etc/logstash create
  • Add Elasticsearch password:
    sudo -E /usr/share/logstash/bin/logstash-keystore --path.settings /etc/logstash add ES_PWD
2

That is telling you that it is trying to read the keystore as part of the configuration. In case this is not clear. If I configure a pipeline using

- pipeline.id: main
  path.config: "/bin/bash"

I get the error message "The following config files contains non-ascii characters but are not UTF-8 encoded "/bin/bash"]"

So when you run as a service path.config is getting set to /etc/logstash. How are you setting path.config when you run from the command line?

3

This wasn't clear to me. So does this mean the location of the keystone file is wrong?

From the command line I set it as a parameter:

sudo -E /usr/share/logstash/bin/logstash --path.settings /etc/logstash -f /etc/logstash/conf.d/my-server01.conf

Another thing I noticed was that /etc/logstash had owner root after installation and it only would start after changing that to logstash:logstash.

4

No, it means the setting of path.config is wrong when your run as a service. path.config can be set using -f on the command line, or in logstash.yml, or, if you are using pipelines, in pipelines.yml.

5

If I set ExecStart in systemd to
ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash" "-f /etc/logstash"
it works like on the command line.

It's still a mystery to me why that is, however.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.