How to run logstash service in the background?

[evacch (Eva Cheong)]

I installed and configured ELK in a CentOS environment.
I have been using the following command to run logstash in the foreground and it works perfectly.

./usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf

Now I want to implement it into the production environment and hence I would like to run the logstash service in the background by which I did tried to run it with

systemctl start logstash

As usual, I always check with the indices with

http://localhost:9200/_cat/indices

But logstash doesn't seem to work if I am not running it in the foreground and there's also no log generated as well if I run with systemctl start logstash.

Another thing is that I have also made some changes to the default logstash.yml file to increase the batch.size, batch.delay and etc but logstash doesn't capture the changes when I run ./usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf.

I have been trying to solve this for the past few days and I need help on this. Thank you so much.

[warkolm (Mark Walkom)]

@evacch

But logstash doesn't seem to work if I am not running it in the foreground and there's also no log generated as well if I run with systemctl start logstash .

There's nothing in /var/log/logstash?

[evacch (Eva Cheong)]

All the logstash logs are stored under /usr/share/logstash/logs and yes there's log found in there when I run using

/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf

but there's nothing when I run using

systemctl start logstash

[warkolm (Mark Walkom)]

If you installed the RPM then the default location is /var/log/logstash.

If you didn't use the RPM, how did you install it?

[evacch (Eva Cheong)]

I run with

sudo yum install logstash

and yes there's such folder /var/log/logstashas configured in the logstash.yml

[warkolm (Mark Walkom)]

Did you add our source repos?
What version did it install?

[evacch (Eva Cheong)]

Yes I did. the current version is 7.8.0.

[evacch (Eva Cheong)]

It works fine now with

./usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf -b 3000 -u 30 &

I am not sure if this is a good practice to start logstash service.

[Jenni]

If it works when you start it like that, but not as a service, it might be a permissions issue, e.g. your log directory being writable by root, but not by the Logstash user.

[evacch (Eva Cheong)]

I have changed all logstash related files and folders permission to logstash user via

chown logstash:logstash folder/file

after that, I run

systemctl stop logstash

then

systemctl start logstash

but it's still the same.

[evacch (Eva Cheong)]

Hello, is there any solution for this? I am stuck for couple days.

[Jenni]

I would use chown -R on the relevant folders to make sure that really no file is forgotten, try it again and check the logstash log directory and syslog. But if it really doesn't say anything anywhere, it's difficult to debug.
Maybe you could run sudo -u logstash /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf. If that works, it's probably something about the config, not the user, that keeps it from working?