Cannot restart(Stop and start) Logstash

Elastic Stack Logstash
1

So I start my logstash conf file, which works great and does exactly what I want it to do. I run iy by manually running: logstash -f csvedit.conf which runs logstash using my .conf file.
I stop this script by pressing cntrl+X, which stops it. However I cannot start it the same way again, as I get the following error:

Logstash could not be started because there is already another instance using the configured data directory. If you wish to run multiple instances, you must change the "path.data" setting.

Any pointer on how I can overcome this problem ?
The ETL job is still in testing, So I want to be able to stop and start logstash for now.

2

You mean, you cannot start again after you stopped or you want to start it and have two or more instances running?

What do you have printed in the screen when you stop it?

Normally Logstash is executed as a service and keeps running.

3

I cannot start again after I stopped it...I have restart my entire Ubuntu VM to run logstash again. (I only have 1 instance running btw)

For some reason I cannot run it as a service. If I do start the logstash service, It does start sucessfully, But It does nothing - in the sense that it does not run my .conf files. Hence, I have resorted to starting it manually, Which has worked great so far Except the fact that I cannot manually start it...After stopping it.

4

What is the exact command line that you are using to start logstash? And which user are you using?

When you stop it does it really stop? Have you checked if the process is really stopped?

I have restart my entire Ubuntu VM to run logstash again

And what does it show when you stop it? Can you restart the VM, run it again and stop to get fresh logs?

Did you configure it in /etc/logstash/pipelines.yml? When you run logstash as a service it will start the pipelines listed in the /etc/logstash/pipelines.yml, you can have multiple pipelines configured.

The error you shared normally happens when Logstash is still running and you are trying to run it again.

5

Indeed. I suspect it suspends it rather than terminates it. Use ps to check if the process still exists.

6

Shouldve mentioned earlier, Thank you firstly for replying. Appreciate!

I start it by running the following: logstash -f csvedit.conf

When I stop and start it again, I get the following:

clinton@master-node:/etc/logstash/conf.d$ logstash -f csvedit.conf
Using bundled JDK: /usr/share/logstash/jdk
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[INFO ] 2025-05-26 23:09:58.351 [main] runner - Starting Logstash {"logstash.version"=>"7.17.28", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.26+4 on 11.0.26+4 +indy +jit [linux-x86_64]"}
[INFO ] 2025-05-26 23:09:59.013 [main] runner - JVM bootstrap flags: [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djdk.io.File.enableADS=true, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -Djruby.regexp.interruptible=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true]
[WARN ] 2025-05-26 23:10:00.802 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[FATAL] 2025-05-26 23:10:00.842 [LogStash::Runner] runner - Logstash could not be started because there is already another instance using the configured data directory. If you wish to run multiple instances, you must change the "path.data" setting.
[FATAL] 2025-05-26 23:10:00.859 [LogStash::Runner] Logstash - Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.20.1.jar:?]
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.20.1.jar:?]
at usr.share.logstash.lib.bootstrap.environment.(/usr/share/logstash/lib/bootstrap/environment.rb:94) ~[?:?]

I have ensured logstash is not running when I do stop it....

I have unfortunately not tried to configure the /etc/logstash/pipelines.yml yet.

7

Can you share what logstash prints to the screen when you successfully start it and stop?

The logs you shared shows that Logstash could not start.

8

So, normally Ctrl-Z suspends a process started from a shell. And Ctrl-C would stop (terminate) it. Ctrl-X, normally, would do nothing. (noting that X is right next to both C and Z on US/UK keyboard)

You dont mention the specific of your environment - linux, which shell, etc.

Your problem now is likely you have a logstash process running, you are going to need to kill it.

ps -ef | fgrep logstash

will help identify the process to kill.

fgrep logstash /proc/*/cmdline

also

pgrep logstash

Er, not sure what this means, but

is pretty clear that logstash thinks another instance is running.

9

Edit: It would also be useful if you add some details on how you installed logstash, which version, and under which operating system please.

10

@RainTown, Thank you so much.
Very embarassingly, I figured the issue out, with your advice.
I have been pressing Cntrl + Z to stop it, and it was actually suspending it.

I have tried Cntrl+C now, and it does indeed stop it, andI can restart it as well.

Thank you for your help.

1 Like