I was having trouble getting Logstash to run correctly when I started it as a service. The logstash user was created correctly but when Logstash was setup as a service no data was reaching our ES cluster and thus not showing up in Kibana. I had a working config when the;
sudo ./logstash --path.settings /etc/logstash -f /etc/logstash/logstash.conf
command was run from the /usr/share/logstash/bin folder so I was perplexed as to why this config wasn't working when we started Logstash as a service.
Eventually I came across this thread in stackoverflow https://stackoverflow.com/questions/38839843/how-to-setup-logstash-so-that-default-configure-file-is-read-when-it-start-up-au/53872270#53872270 and this was our problem.
Our fix was to check the permissions of the appropriate folders by using the command ll and saw that the folder had root permissions. Then we changed the permissions for our /etc/logstash and /usr/share/logstash folders. Used the command;
sudo chown -R logstash:logstash logstash
while in the /etc folder. For our /usr/share/logstash folder there was only one file that didnt have the right permissions so we used the cmd
sudo chown logstash:logstash <file_name>
to change the permissions for that file. Everything booted correctly when we used the sudo systemctl start logstash command after the permissions were updated. Hope this helps someone else that might have the same problem.