Trying to get multiple pipelines work from logstash and fed up with settings. Below are details
logstash.yml
path.data: /var/lib/logstash
path.logs: /var/log/logstash
Note that "path.config" is not present as I want logstash to pickup pipelines.yml file. pipelines.yml file is present in the same directory as logstash.yml.
pipelines.yml
-
pipeline.id: system-messages-devtest
path.config: "/etc/logstash/conf.d/ls-system.messages.pipeline.yml" -
pipeline.id: system-auth-devtest
path.config: "/etc/logstash/conf.d/ls-system.auth.pipeline.yml"
NOTE: It will resemble like filebeat name as I re-used a bit from there for transformations.
Logstash is started using "sudo systemctl start logstash.service"
Also, tried with sudo /usr/share/logstash/bin/logstash --path.settings /etc/logstash
Error:
[root@........]# sudo /usr/share/logstash/bin/logstash --path.settings /etc/logstash
Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties
ERROR: No configuration file was specified. Perhaps you forgot to provide the '-f yourlogstash.conf' flag?
usage:
bin/logstash -f CONFIG_PATH [-t] [-r] [] [-w COUNT] [-l LOG]
bin/logstash --modules MODULE_NAME [-M "MODULE_NAME.var.PLUGIN_TYPE.PLUGIN_NAME.VARIABLE_NAME=VALUE"] [-t] [-w COUNT] [-l LOG]
bin/logstash -e CONFIG_STR [-t] [--log.level fatal|error|warn|info|debug|trace] [-w COUNT] [-l LOG]
bin/logstash -i SHELL [--log.level fatal|error|warn|info|debug|trace]
bin/logstash -V [--log.level fatal|error|warn|info|debug|trace]
bin/logstash --help
When started as service using "sudo systemctl start logstash.service", below is observed in log /var/log/logstash/logstash-plain.log
[root@........]# sudo systemctl start logstash.service
[root@........]# ps -ef|grep logstash
logstash 16770 1 99 22:42 ? 00:00:28 /usr/bin/java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -Djava.awt.headless=true -Dfile.encoding=UTF-8 -XX:+HeapDumpOnOutOfMemoryError -Xmx1g -Xms256m -Xss2048k -Djffi.boot.library.path=/usr/share/logstash/vendor/jruby/lib/jni -Xbootclasspath/a:/usr/share/logstash/vendor/jruby/lib/jruby.jar -classpath : -Djruby.home=/usr/share/logstash/vendor/jruby -Djruby.lib=/usr/share/logstash/vendor/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh org.jruby.Main /usr/share/logstash/lib/bootstrap/environment.rb logstash/runner.rb --path.settings /etc/logstash
root 16808 10416 0 22:42 pts/0 00:00:00 grep --color=auto logstash
[root@........]# cat /var/log/logstash/logstash-plain.log
[2018-01-16T22:43:19,202][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"}
[2018-01-16T22:43:19,220][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"}
NO PIPELINES ARE STARTED
Any help is appreciated.