Hi I have configure Elasticsearch, kibana & logstash on same machine. Want to receive firewall logs in logstash but facing below error. For now want to show these event on console only once received will forward in elastic.
/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/firewall.conf
Using bundled JDK: /usr/share/logstash/jdk
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[INFO ] 2022-09-13 08:34:39.095 [main] runner - Starting Logstash {"logstash.version"=>"8.4.1", "jruby.version"=>"jruby 9.3.6.0 (2.6.8) 2022-06-27 7a2cbcd376 OpenJDK 64-Bit Server VM 17.0.4+8 on 17.0.4+8 +indy +jit [x86_64-linux]"}
[INFO ] 2022-09-13 08:34:39.103 [main] runner - JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED]
Your settings are invalid. Reason: Path "/usr/share/logstash/data/queue" must be a writable directory. It is not writable.
[FATAL] 2022-09-13 08:34:39.142 [main] Logstash - Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:790) ~[jruby.jar:?]
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:753) ~[jruby.jar:?]
at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:91) ~[?:?]
**firewall Configuration file is provided below:**
# Input plugin that will receive the logs on port 5144
# on the specified host interface IP
######################################
input {
udp {
host => "172.16.41.151"
port => 5140
}
}
# Filter plugins that will modify the logs
##########################################
filter {
grok {
match => {"message" => "%{SYSLOG5424PRI}%{GREEDYDATA:message}" }
overwrite => [ "message" ]
}
mutate {
remove_field => ["@timestamp","host","@version","event","log"]
}
kv {
field_split => " "
}
mutate {
remove_field => ["message"]
add_field => { "logdate" => "%{date} %{time}" }
}
date {
match => [ "logdate", "yyyy-MM-dd HH:mm:ss" ]
timezone => "America/Edmonton"
target => "@timestamp"
}
mutate {
remove_field => ["logdate","date","time"]
convert => { "rcvdbyte" => "integer" }
convert => { "sentbyte" => "integer" }
}
}
# Output plugin to send the logs to Elasticsearch
#################################################
output {
stdout {} # Use stdout to see the output on the console during testing
elasticsearch {
after running with specified path.settings receiving this error
Using bundled JDK: /usr/share/logstash/jdk
Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties
2022-09-13 09:12:08,544 main ERROR RollingFileManager (/var/log/logstash/logstash-plain.log) java.io.FileNotFoundException: /var/log/logstash/logstash-plain.log (Permission denied) java.io.FileNotFoundException: /var/log/logstash/logstash-plain.log (Permission denied)
at java.base/java.io.FileOutputStream.open0(Native Method)
at java.base/java.io.FileOutputStream.open(FileOutputStream.java:293)
at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:235)
at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:155)
at org.apache.logging.log4j.core.appender.rolling.RollingFileManager$RollingFileManagerFactory.createManager(RollingFileManager.java:746)
at org.apache.logging.log4j.core.appender.rolling.RollingFileManager$RollingFileManagerFactory.createManager(RollingFileManager.java:716)
at org.apache.logging.log4j.core.appender.AbstractManager.getManager(AbstractManager.java:114)
at org.apache.logging.log4j.core.appender.OutputStreamManager.getManager(OutputStreamManager.java:100)
at org.apache.logging.log4j.core.appender.rolling.RollingFileManager.getFileManager(RollingFileManager.java:217)
at org.apache.logging.log4j.core.appender.RollingFileAppender$Builder.build(RollingFileAppender.java:146)
at org.apache.logging.log4j.core.appender.RollingFileAppender$Builder.build(RollingFileAppender.java:62)
at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:122)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:1120)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:1045)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:1037)
at org.apache.logging.log4j.core.config.AbstractConfiguration.doConfigure(AbstractConfiguration.java:651)
at org.apache.logging.log4j.core.config.AbstractConfiguration.initialize(AbstractConfiguration.java:247)
at org.logstash.log.LogstashConfigurationFactory.getConfiguration(LogstashConfigurationFactory.java:68)
at org.logstash.log.LogstashConfigurationFactory.getConfiguration(LogstashConfigurationFactory.java:40)
at org.apache.logging.log4j.core.config.ConfigurationFactory.getConfiguration(ConfigurationFactory.java:302)
at org.apache.logging.log4j.core.config.ConfigurationFactory$Factory.getConfiguration(ConfigurationFactory.java:465)
at org.apache.logging.log4j.core.config.ConfigurationFactory.getConfiguration(ConfigurationFactory.java:323)
at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:695)
at org.apache.logging.log4j.core.LoggerContext.setConfigLocation(LoggerContext.java:684)
at org.logstash.log.LoggerExt.reconfigure(LoggerExt.java:184)
at org.logstash.log.LoggerExt$INVOKER$s$1$0$reconfigure.call(LoggerExt$INVOKER$s$1$0$reconfigure.gen)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:372)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:175)
at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:316)
at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
at org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:116)
at org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:136)
at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:66)
at org.jruby.runtime.Block.call(Block.java:147)
at org.jruby.RubyProc.call(RubyProc.java:272)
at org.jruby.RubyProc$INVOKER$i$call.call(RubyProc$INVOKER$i$call.gen)
at org.jruby.ir.targets.indy.InvokeSite.invoke(InvokeSite.java:208)
at usr.share.logstash.logstash_minus_core.lib.logstash.settings.RUBY$block$post_process$1(/usr/share/logstash/logstash-core/lib/logstash/settings.rb:198)
2022-09-13 09:12:08,608 main ERROR Null object returned for RollingFile in Appenders.
2022-09-13 09:12:08,608 main ERROR Null object returned for RollingFile in Appenders.
2022-09-13 09:12:08,609 main ERROR Unable to locate appender "plain_rolling" for logger config "root"
2022-09-13 09:12:08,609 main ERROR Unable to locate appender "plain_rolling_slowlog" for logger config "slowlog"
2022-09-13 09:12:08,610 main ERROR Unable to locate appender "deprecation_plain_rolling" for logger config "org.logstash.deprecation, deprecation"
2022-09-13 09:12:08,610 main ERROR Unable to locate appender "deprecation_plain_rolling" for logger config "deprecation"
[2022-09-13T09:12:08,631][INFO ][logstash.runner ] Log4j configuration path used is: /etc/logstash/log4j2.properties
[2022-09-13T09:12:08,642][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"8.4.1", "jruby.version"=>"jruby 9.3.6.0 (2.6.8) 2022-06-27 7a2cbcd376 OpenJDK 64-Bit Server VM 17.0.4+8 on 17.0.4+8 +indy +jit [x86_64-linux]"}
[2022-09-13T09:12:08,644][INFO ][logstash.runner ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED]
Your settings are invalid. Reason: Path "/var/lib/logstash" must be a writable directory. It is not writable.
[2022-09-13T09:12:08,664][FATAL][org.logstash.Logstash ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:790) ~[jruby.jar:?]
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:753) ~[jruby.jar:?]
at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:91) ~[?:?]
[qds@localhost ~]$
Yes, I have run this as a root/sudo user, and have installed logstash as a service.
[root@localhost ~]# /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/firewall.conf
Using bundled JDK: /usr/share/logstash/jdk
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[WARN ] 2022-09-13 09:27:14.266 [main] runner - NOTICE: Running Logstash as superuser is not recommended and won't b e allowed in the future. Set 'allow_superuser' to 'false' to avoid startup errors in future releases.
[INFO ] 2022-09-13 09:27:14.281 [main] runner - Starting Logstash {"logstash.version"=>"8.4.1", "jruby.version"=>"jr uby 9.3.6.0 (2.6.8) 2022-06-27 7a2cbcd376 OpenJDK 64-Bit Server VM 17.0.4+8 on 17.0.4+8 +indy +jit [x86_64-linux]"}
[INFO ] 2022-09-13 09:27:14.284 [main] runner - JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfi le.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -XX:+HeapDumpOnOutOfMemoryError, -Dj ava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.comp iler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add- exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-U NNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java .base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.m anagement=ALL-UNNAMED]
[WARN ] 2022-09-13 09:27:14.589 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[FATAL] 2022-09-13 09:27:14.596 [LogStash::Runner] runner - Logstash could not be started because there is already a nother instance using the configured data directory. If you wish to run multiple instances, you must change the "pa th.data" setting.
[FATAL] 2022-09-13 09:27:14.599 [LogStash::Runner] Logstash - Logstash stopped processing because of an error: (Syst emExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:790) ~[jruby.jar:?]
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:753) ~[jruby.jar:?]
at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:
[FATAL] 2022-09-13 09:27:14.596 [LogStash::Runner] runner - Logstash could not be started because there is already another instance using the configured data directory. If you wish to run multiple instances, you must change the "path.data" setting.
Are you trying to run Logstash as a service, with systemctl?
Your main issue is that you run Logstash as the root user which is not recommend because it will break the permissions if you try to run it later as a service.
If you want to run logstash as a service using systemctl you will need to fix the permissions for the logstash directories.
For this you will need to stop all the instances you are running and start it as a service, look at the logs and see what file or path it does not have permission and then fix the permission.
It is confusing now, if you are already running Logstash as a service you should not try to run it again in the same server using the command line.
If the service is not running, please share the logs from the service, all the logs you shared before were from Logstash running from the command line.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.