private proprietary enterprise professional background developing on Open Source.. what a pain in the neck!
new into this nightmare, ive spent around 36 hrs fighting new ELK install ( from ubuntu ISO to application ) Setup from scratch ELK installed on ubuntu 1804 - had lots of issues on the way until get to the below error: mainly related to Logstash permissions, settings etg. I have read and learn on the way so much but I need help troubleshooting this:
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:5601 0.0.0.0:* LISTEN 862/node
tcp 0 0 127.0.0.1:9200 0.0.0.0:* LISTEN 965/java
tcp 0 0 127.0.0.1:9300 0.0.0.0:* LISTEN 965/java
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 801/systemd-resolve
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 950/sshd
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 22962/cupsd
tcp6 0 0 :::22 :::* LISTEN 950/sshd
udp 0 0 0.0.0.0:39582 0.0.0.0:* 824/avahi-daemon: r
udp 0 0 127.0.0.53:53 0.0.0.0:* 801/systemd-resolve
udp 0 0 0.0.0.0:631 0.0.0.0:* 22963/cups-browsed
udp 0 0 0.0.0.0:5353 0.0.0.0:* 824/avahi-daemon: r
udp6 0 0 :::48471 :::* 824/avahi-daemon: r
udp6 0 0 :::5353 :::* 824/avahi-daemon: r
Who the hell is avahi-daemon?
root@adminri-virtual-machine:/usr/share/logstash/bin# sudo /usr/share/logstash/bin/logstash
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/usr/share/logstash/logstash-core/lib/jars/jruby-complete-188.8.131.52.jar) to method sun.nio.ch.NativeThread.signal(long)
WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
ERROR: Failed to read pipelines yaml file. Location: /usr/share/logstash/config/pipelines.yml
bin/logstash -f CONFIG_PATH [-t] [-r] [-w COUNT] [-l LOG]
bin/logstash --modules MODULE_NAME [-M "MODULE_NAME.var.PLUGIN_TYPE.PLUGIN_NAME.VARIABLE_NAME=VALUE"] [-t] [-w COUNT] [-l LOG]
bin/logstash -e CONFIG_STR [-t] [--log.level fatal|error|warn|info|debug|trace] [-w COUNT] [-l LOG]
bin/logstash -i SHELL [--log.level fatal|error|warn|info|debug|trace]
bin/logstash -V [--log.level fatal|error|warn|info|debug|trace]
[ERROR] 2020-04-23 00:40:35.356 [LogStash::Runner] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
I dont really understand why sometimes I run logstash it gets run from different paths: /etc/logstash/ . It seems the reason resides when I first downloaded and installed the package it didnt come from elastic but other repo instead mixing and messing the execution, configurational, settings paths.
It did happen the same on FreeBSD where I run my prod pfsense and installed Beats for Freebsd- I had to be very careful from where do I run the service ( path ) .
root@adminri-virtual-machine:/usr/share/logstash/bin# sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/ --path.settings=/etc/logstash --log.level=debug
[2020-04-23T01:00:53,816][DEBUG][logstash.runner ] --------------- Logstash Settings -------------------
[2020-04-23T01:00:53,868][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2020-04-23T01:00:53,884][FATAL][logstash.runner ] Logstash could not be started because there is already another instance using the configured data directory. If you wish to run multiple instances, you must change the "path.data" setting.
[2020-04-23T01:00:53,893][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
And another observation copying and paste from to terminal modify certain characters, a missing 'double quote' here, extra # there ... 0 consistency between applications I dont see where the efficiencies are
thanks to who ever picks up this thread.. Im new to the tech so potentially a tech controbutor.