Logstash unable to run conf file in background?

Elastic Stack Logstash
1

Hi there, how can I run the logstash .conf file in background ?
I am using Centos 6.9 and I installed logstash with rpm package.

file: /etc/logstash/logstash.yml

pipeline.id: logstash_for_nagios
path.config: "/etc/logstash/conf.d/nagios.conf"
pipeline.workers: 1

file: /etc/logstash/pipelines.yml

- pipeline.id: main
  path.config: "/etc/logstash/conf.d/nagios.conf"
  pipeline.worker: 2

and I set the user and group to root

file: /etc/logstash/startup.options

LS_USER=root
LS_GROUP=root

with the ps aux | grep logstash command. Below is the result:

[root@casrh6t logstash]# ps aux | grep logstash
logstash  32095 83.8  9.1 3088564 368480 ?      SNsl 11:10   0:31 /usr/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djruby.compile.invokedynamic=true -Djruby.jit.threshold=0 -Djruby.regexp.interruptible=true -XX:+HeapDumpOnOutOfMemoryError -Djava.security.egd=file:/dev/urandom -Dlog4j2.isThreadContextMapInheritable=true -cp /usr/share/logstash/logstash-core/lib/jars/animal-sniffer-annotations-1.14.jar:/usr/share/logstash/logstash-core/lib/jars/commons-codec-1.11.jar:/usr/share/logstash/logstash-core/lib/jars/commons-compiler-3.0.11.jar:/usr/share/logstash/logstash-core/lib/jars/error_prone_annotations-2.0.18.jar:/usr/share/logstash/logstash-core/lib/jars/google-java-format-1.1.jar:/usr/share/logstash/logstash-core/lib/jars/gradle-license-report-0.7.1.jar:/usr/share/logstash/logstash-core/lib/jars/guava-22.0.jar:/usr/share/logstash/logstash-core/lib/jars/j2objc-annotations-1.1.jar:/usr/share/logstash/logstash-core/lib/jars/jackson-annotations-2.9.10.jar:/usr/share/logstash/logstash-core/lib/jars/jackson-core-2.9.10.jar:/usr/share/logstash/logstash-core/lib/jars/jackson-databind-2.9.10.1.jar:/usr/share/logstash/logstash-core/lib/jars/jackson-dataformat-cbor-2.9.10.jar:/usr/share/logstash/logstash-core/lib/jars/janino-3.0.11.jar:/usr/share/logstash/logstash-core/lib/jars/javassist-3.24.0-GA.jar:/usr/share/logstash/logstash-core/lib/jars/jruby-complete-9.2.8.0.jar:/usr/share/logstash/logstash-core/lib/jars/jsr305-1.3.9.jar:/usr/share/logstash/logstash-core/lib/jars/log4j-api-2.11.1.jar:/usr/share/logstash/logstash-core/lib/jars/log4j-core-2.11.1.jar:/usr/share/logstash/logstash-core/lib/jars/log4j-slf4j-impl-2.11.1.jar:/usr/share/logstash/logstash-core/lib/jars/logstash-core.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.core.commands-3.6.0.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.core.contenttype-3.4.100.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.core.expressions-3.4.300.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.core.filesystem-1.3.100.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.core.jobs-3.5.100.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.core.resources-3.7.100.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.core.runtime-3.7.0.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.equinox.app-1.3.100.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.equinox.common-3.6.0.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.equinox.preferences-3.4.1.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.equinox.registry-3.5.101.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.jdt.core-3.10.0.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.osgi-3.7.1.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.text-3.5.101.jar:/usr/share/logstash/logstash-core/lib/jars/reflections-0.9.11.jar:/usr/share/logstash/logstash-core/lib/jars/slf4j-api-1.7.25.jar org.logstash.Logstash --path.settings /etc/logstash
root      32647  0.0  0.0 103332   892 pts/3    S+   11:10   0:00 grep logstash
[root@casrh6t logstash]#

and my /etc/logstash/conf.d/nagios.conf file which I want to run forever in the background

input {
  file {
    path => ["/usr/local/nagios/var/nagios.log"]
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}
filter {
// FILTERING
}
output {
  elasticsearch {
    hosts => ["http://10.61.35.162:9200/"]
    index => "test-nagios"
  }
  stdout { codec => rubydebug }
}
2

What do the Logstash logs show?

3

sorry. Where is log file located at ??

4

Sorry I found my log file located and here is the latest log msg.

[2020-03-09T12:06:56,915][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<ArgumentError: Path "/var/lib/logstash/queue" must be a writable directory. It is not writable.>, :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/settings.rb:489:in validate'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:271:in validate_value'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:182:in block in validate_all'", "org/jruby/RubyHash.java:1417:in each'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:181:in validate_all'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:284:in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in run'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:242:in run'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in run'", "/usr/share/logstash/lib/bootstrap/environment.rb:73:in '"]}
[2020-03-09T12:06:57,341][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

5

I have tried /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/nagios.conf & command but it does not work once I closed the terminal.

6

Hi

This might be an issue:

Do you run logstash as root?

Hope this helps.

7

Yes.. I run logstash as root.

8

Up.. I still need help.

Followed from https://www.elastic.co/guide/en/logstash/current/running-logstash.html#running-logstash-upstart

but it does not work too.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.