Further:
The service file is here: /etc/systemd/system/logstash.service
It contains the following which is all default:
[Unit]
Description=logstash
[Service]
Type=simple
User=logstash
Group=logstash
EnvironmentFile=-/etc/default/logstash
EnvironmentFile=-/etc/sysconfig/logstash
ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash"
Restart=always
WorkingDirectory=/
Nice=19
LimitNOFILE=16384
[Install]
WantedBy=multi-user.target
It points to an EnvironmentFile here: /etc/default/logstash
...which contains the following:
JAVACMD="/usr/bin/java"
LS_HOME="/usr/share/logstash"
LS_SETTINGS_DIR="/etc/logstash"
LS_PIDFILE="/var/run/logstash.pid"
LS_USER="logstash"
LS_GROUP="logstash"
LS_GC_LOG_FILE="/var/log/logstash/gc.log"
LS_OPEN_FILES="16384"
LS_NICE="19"
SERVICE_NAME="logstash"
SERVICE_DESCRIPTION="logstash"
So it has all the correct paths, all default, but when I start Logstash as a service it doesn't find the settings file, and hence no logging and no config file.
Starting it like this from /usr/share/logstash works:
bin/logstash --debug --path.settings /etc/logstash/
When I start it manually and specify the settings file I am doing so as root rather than the logstash user, but the /etc/logstash directory is readable by everyone
drwxrwxr-x 2 root root 71 May 12 09:42 conf.d
-rw-r--r-- 1 root root 1738 Apr 28 14:15 jvm.options
-rw-r--r-- 1 root root 1334 Apr 28 14:15 log4j2.properties
-rw-r--r-- 1 root root 223 May 11 22:40 logrotate.logstash.conf
-rw-r--r-- 1 root root 4483 May 12 13:49 logstash.yml
-rw-r--r-- 1 root root 1659 Apr 28 14:15 startup.options