ERROR: Failed to read pipelines yaml file. Location: /usr/share/logstash/config/pipelines.yml

private proprietary enterprise professional background developing on Open Source.. what a pain in the neck!

new into this nightmare, ive spent around 36 hrs fighting new ELK install ( from ubuntu ISO to application ) Setup from scratch ELK installed on ubuntu 1804 - had lots of issues on the way until get to the below error: mainly related to Logstash permissions, settings etg. I have read and learn on the way so much but I need help troubleshooting this:
netstat -plnt

Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0* LISTEN 862/node
tcp 0 0* LISTEN 965/java
tcp 0 0* LISTEN 965/java
tcp 0 0* LISTEN 801/systemd-resolve
tcp 0 0* LISTEN 950/sshd
tcp 0 0* LISTEN 22962/cupsd
tcp6 0 0 :::22 :::* LISTEN 950/sshd
udp 0 0* 824/avahi-daemon: r
udp 0 0* 801/systemd-resolve
udp 0 0* 22963/cups-browsed
udp 0 0* 824/avahi-daemon: r
udp6 0 0 :::48471 :::* 824/avahi-daemon: r
udp6 0 0 :::5353 :::* 824/avahi-daemon: r

Who the hell is avahi-daemon?
root@adminri-virtual-machine:/usr/share/logstash/bin# sudo /usr/share/logstash/bin/logstash
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/usr/share/logstash/logstash-core/lib/jars/jruby-complete- to method
WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/ Using default config which logs errors to the console
ERROR: Failed to read pipelines yaml file. Location: /usr/share/logstash/config/pipelines.yml
bin/logstash -f CONFIG_PATH [-t] [-r] [-w COUNT] [-l LOG]
bin/logstash -e CONFIG_STR [-t] [--log.level fatal|error|warn|info|debug|trace] [-w COUNT] [-l LOG]
bin/logstash -i SHELL [--log.level fatal|error|warn|info|debug|trace]
bin/logstash -V [--log.level fatal|error|warn|info|debug|trace]
bin/logstash --help
[ERROR] 2020-04-23 00:40:35.356 [LogStash::Runner] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

I dont really understand why sometimes I run logstash it gets run from different paths: /etc/logstash/ . It seems the reason resides when I first downloaded and installed the package it didnt come from elastic but other repo instead mixing and messing the execution, configurational, settings paths.
A mess.
It did happen the same on FreeBSD where I run my prod pfsense and installed Beats for Freebsd- I had to be very careful from where do I run the service ( path ) .


root@adminri-virtual-machine:/usr/share/logstash/bin# sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/ --path.settings=/etc/logstash --log.level=debug

[2020-04-23T01:00:53,816][DEBUG][logstash.runner ] --------------- Logstash Settings -------------------
[2020-04-23T01:00:53,868][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2020-04-23T01:00:53,884][FATAL][logstash.runner ] Logstash could not be started because there is already another instance using the configured data directory. If you wish to run multiple instances, you must change the "" setting.
[2020-04-23T01:00:53,893][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

And another observation copying and paste from to terminal modify certain characters, a missing 'double quote' here, extra # there ... 0 consistency between applications I dont see where the efficiencies are

thanks to who ever picks up this thread.. Im new to the tech so potentially a tech controbutor.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.