Logstash crash when starting after messing the queue files

Elastic Stack Logstash
1

I've just upgraded Logstash minor version from 7.13 to 7.17.9, but it doesn't restart, erroring about inability to create queues (there was a forced stop of Logstash, so I assume the files are corrupted).
I tried to delete the queue folders:
/usr/share/logstash/data/prod/queue/
Which was a big mistake... and now it errors the following:

[2023-02-04T11:08:52,519][INFO ][logstash.runner          ] Log4j configuration path used is: /etc/logstash/log4j2.properties
[2023-02-04T11:08:52,530][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.17.9", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.18+10 on 11.0.18+10 +indy +jit [linux-x86_64]"}
[2023-02-04T11:08:52,533][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms4g, -Xmx16g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -Djruby.regexp.interruptible=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true]
[2023-02-04T11:08:52,571][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.20.1.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.20.1.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:94) ~[?:?]

How can I restore logstash, should I delete the data folder completely ?

2

This may indicate a configuration error, did you change anything else?

Check the systems log for any hint of why the service is not starting, look at /var/log/messages or /var/log/syslog depending on the distribution.

3

Thanks for your answer.

There was no change to the config files, just the upgrade.
I ended making a VM snapshot and deleting the full data/prod/ folder and it solved the issue.

However I get a lot of issues with the tracking metadata, I'll open another topic on that.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.