[logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or co mmand line options are specified

[lug-gh]

Hi, I wan't to use multiple pipelines, but after some testing I noticed the pipelines.yml won't be loaded. I searched the web and found it's ignored when you specify -f or -e (Multiple Pipelines | Logstash Reference [8.11] | Elastic)

I don't use that paramenter,

removed wrong information

but by default the systemd file includes path.settings, which is the same as -f (Running Logstash from the Command Line | Logstash Reference [8.11] | Elastic)

ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash"

So I modified my /etc/logstash/startup.options and commented the line LS_OPTS="--path.settings ${LS_SETTINGS_DIR}" and executed /usr/share/logstash/bin/system-install

After that the --path.settings was removed from the systemd-file, but after a restart of logstash, it still says it's ignoring the file...

How can I resolve this issue?

[Badger]

@lug-gh

by default the systemd file includes path.settings, which is the same as -f

No, -f is path.config, not path.settings.

[lug-gh]

@Badger

path.settings

Ah my mistake, thank you
But anyway, the problem is still the same

[leandrojmp (Leandro Pereira)]

Please share the logstash log after you start it with systemctl start logstash.

The ExecStart line in the service file should look like this:

ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash"

[lug-gh]

cat /etc/systemd/system/logstash.service

[Unit]
Description=logstash

[Service]
Type=simple
User=root
Group=root
# Load env vars from /etc/default/ and /etc/sysconfig/ if they exist.
# Prefixing the path with '-' makes it try to load, but if the file doesn't
# exist, it continues onward.
EnvironmentFile=-/etc/default/logstash
EnvironmentFile=-/etc/sysconfig/logstash
ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash"
Restart=always
WorkingDirectory=/
Nice=19
LimitNOFILE=16384

# When stopping, how long to wait before giving up and sending SIGKILL?
# Keep in mind that SIGKILL on a process can cause data loss.
TimeoutStopSec=infinity

[Install]
WantedBy=multi-user.target

systemctl restart logstash.service

[2022-03-29T08:25:40,813][INFO ][logstash.runner          ] Log4j configuration path used is: /etc/logstash/log4j2.properties
[2022-03-29T08:25:40,821][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.17.1", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.13+8 on 11.0.13+8 +indy +jit [linux-x86_64]"}
[2022-03-29T08:25:40,823][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, -Xmx4g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -Djruby.regexp.interruptible=true, -XX:-HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true]
[2022-03-29T08:25:41,071][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2022-03-29T08:25:41,921][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}

[leandrojmp (Leandro Pereira)]

This is the entire log?

If it is not using pipelines.yml it needs to run another config or it will crash, do you have any other logs in the file? You need to have something else in the log, it will show that logstash stopped because it has no configs to run or it will show the config it is running.

But something is wrong because your service file is correct, except for the user, why did you change the user and group to root? It is not recommended to run logstash as root.

Also, did you run systemctl daemon-reload after changing the service file?

[lug-gh]

I think I just found the issue.. my logstash.yml contains
path.config: /etc/logstash/conf.d, which is why the pipelines.yml is ignored....
Thanks anyway!