Hello,
I'm getting the following error when I try to start a newly configured logstash instance:
[2018-08-13T17:17:04,939][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
[2018-08-13T17:17:19,214][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<TypeError: no implicit conversion of nil into String>, :backtrace=>["org/jruby/RubyFileTest.java:96:in `directory?'", "org/jruby/RubyFileTest.java:88:in `directory?'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:469:in `block in value'", "org/jruby/RubyKernel.java:1741:in `tap'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:468:in `value'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:230:in `validate_value'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:141:in `block in validate_all'", "org/jruby/RubyHash.java:1343:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:140:in `validate_all'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:279:in `execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:238:in `run'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in `run'", "/usr/share/logstash/lib/bootstrap/environment.rb:73:in `<main>'"]}
[2018-08-13T17:17:19,227][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
I've used the latest RPM from the Elastic Repo:
# rpm -qa |grep logstash
logstash-6.3.1-1.noarch
My /etc/logstash/logstash.yml (partly obfuscated):
# grep -v "^#" /etc/logstash/logstash.yml
node.name: XXXXp66
path.data: /var/lib/logstash/
path.config: /etc/logstash/conf.d/*.conf
queue.type: persisted
path.queue: /var/lib/logstash/queue
queue.page_capacity: 64mb
queue.max_bytes: 80gb
queue.checkpoint.acks: 1024
queue.checkpoint.writes: 1024
queue.checkpoint.interval: 1000
dead_letter_queue.enable: true
dead_letter_queue.max_bytes: 1024mb
path.dead_letter_queue:
http.host: "[XXXX:XXXX:X:9::2]"
http.port: 9600
path.logs: /var/log/logstash
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: logstash_system
xpack.monitoring.elasticsearch.password: ${logstash_system_password}
xpack.monitoring.elasticsearch.url: ["https://[XXXX:XXXX:X:a::5]:9200", "https://[XXXX:XXXX:X:a::6]:9200", "[XXXX:XXXX:X:a::7]:9200"]
xpack.monitoring.elasticsearch.ssl.ca: "/etc/logstash/certs/XXXXp66.ca.crt"
xpack.monitoring.elasticsearch.ssl.verification_mode: certificate
xpack.monitoring.elasticsearch.sniffing: false
xpack.monitoring.collection.interval: 10s
xpack.monitoring.collection.pipeline.details.enabled: true
xpack.management.enabled: false
xpack.management.pipeline.id: ["..." ]
xpack.management.elasticsearch.username: logstash_system
xpack.management.elasticsearch.password: ${logstash_system_password}
xpack.management.elasticsearch.url: ["https://[XXXX:XXXX:X:a::5]:9200", "https://[XXXX:XXXX:X:a::6]:9200", "[XXXX:XXXX:X:a::7]:9200"]
xpack.management.elasticsearch.ssl.ca: "/etc/logstash/certs/XXXXp66.ca.crt"
xpack.management.elasticsearch.sniffing: false
xpack.management.logstash.poll_interval: 5s
Contents of /var/lib/logstash/ directory:
# ls -laZ /var/lib/logstash/
drwxr-xr-x. logstash logstash system_u:object_r:var_lib_t:s0 .
drwxr-xr-x. root root system_u:object_r:var_lib_t:s0 ..
drwxr-xr-x. logstash logstash system_u:object_r:var_lib_t:s0 queue
I've found some previous threads, where data.path has not been set, but that doesn't seem the case here.
Anyone any idea?
Thanks!