Problems to start Logstash conf pipelines with SystemCTL

Elastic Stack Logstash
1

I have a problem with logstash, when I running like this:/usr/share/logstash/bin/logstash --debug -f /etc/logstash/conf.d/switch.conf
Works fine and start this specific configuration file.

But I put this conf file in pipelines.yml, and when I tried to run my logstash through systemctl start logstash.service , appears status OK, but the configuration switch.conf not starts. Another strange behavior I`m see, that the log files folder /var/log/logstash still empty with no logs.

The config files are bellow:

cat /etc/logstash/logstash.yml |grep -v ^#

path.data: /var/lib/logstash
pipeline.ordered: auto

path.logs: /var/log/logstash

cat pipelines.yml

This file is where you define your pipelines. You can define multiple.

For more information on multiple pipelines, see the documentation:

Multiple Pipelines | Logstash Reference [7.14] | Elastic

  • pipeline.id: main
    path.config: "/etc/logstash/conf.d/*.conf"
  • pipeline.id: switch
    path.config: "/etc/logstash/conf.d/switch.conf"

cat /etc/systemd/system/logstash.service

[Unit]
Description=logstash

[Service]
Type=simple
User=logstash
Group=logstash

Load env vars from /etc/default/ and /etc/sysconfig/ if they exist.

Prefixing the path with '-' makes it try to load, but if the file doesn't

exist, it continues onward.

EnvironmentFile=-/etc/default/logstash
EnvironmentFile=-/etc/sysconfig/logstash
ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash"
Restart=always
WorkingDirectory=/
Nice=19
LimitNOFILE=16384

When stopping, how long to wait before giving up and sending SIGKILL?

Keep in mind that SIGKILL on a process can cause data loss.

TimeoutStopSec=infinity

[Install]
WantedBy=multi-user.target

cat /etc/default/logstash

LS_HOME="/usr/share/logstash"
LS_SETTINGS_DIR="/etc/logstash"
LS_PIDFILE="/var/run/logstash.pid"
LS_USER="logstash"
LS_GROUP="logstash"
LS_GC_LOG_FILE="/var/log/logstash/gc.log"
LS_OPEN_FILES="16384"
LS_NICE="19"
SERVICE_NAME="logstash"
SERVICE_DESCRIPTION="logstash"

ls -la /var/log/logstash/

total 8
drwxr-xr-x 2 logstash root 4096 jul 29 16:45 .
drwxr-xr-x 11 root root 4096 ago 29 00:00 ..

2

This is your pipelines.yml ?

- pipeline.id: main
  path.config: "/etc/logstash/conf.d/*.conf"
- pipeline.id: switch
  path.config: "/etc/logstash/conf.d/switch.conf"

If so, this is wrong, it will make logstash read the switch.conf twice, the main pipeline will read all the .conf files inside the conf.d directory.

If your switch.conf pipeline is listening on some port, it will not run when logstash tries to start the switch pipeline as it would already be listening when the main pipeline was started.

Remove the first two lines and try to use this pipelines.yml

- pipeline.id: switch
  path.config: "/etc/logstash/conf.d/switch.conf"
3

Thank you.... works fine!!!!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.