Logstash service stops every 42 seconds and restarts

Elastic Stack Logstash
1

I have all ownership in /etc/logstash/ set to logstash:logstash
I have all ownership in /usr/share/logstash set to logstash:logstash

The logstash service runs as logstash. When I run logstash from the /usr/share/logstash/bin manually everything works fine. When I run it from the service I get these errors.
Nothing in logs

May 02 18:25:11 librenms logstash[18366]:         at org.logstash.Logstash.run(Logstash.java:81)
May 02 18:25:11 librenms logstash[18366]:         at org.logstash.Logstash.main(Logstash.java:45)
May 02 18:25:11 librenms logstash[18366]: 2018-05-02 18:25:11,246 main ERROR Null object returned for RollingFile in Appenders.
May 02 18:25:11 librenms logstash[18366]: 2018-05-02 18:25:11,247 main ERROR Null object returned for RollingFile in Appenders.
May 02 18:25:11 librenms logstash[18366]: 2018-05-02 18:25:11,248 main ERROR Null object returned for RollingFile in Appenders.
May 02 18:25:11 librenms logstash[18366]: 2018-05-02 18:25:11,248 main ERROR Null object returned for RollingFile in Appenders.
May 02 18:25:11 librenms logstash[18366]: 2018-05-02 18:25:11,249 main ERROR Unable to locate appender "plain_rolling" for logger config "root"
May 02 18:25:11 librenms logstash[18366]: 2018-05-02 18:25:11,250 main ERROR Unable to locate appender "plain_rolling_slowlog" for logger config "slowlog"
May 02 18:25:11 librenms logstash[18366]: [2018-05-02T18:25:11,321][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configurat
May 02 18:25:11 librenms logstash[18366]: [2018-05-02T18:25:11,356][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/config
May 02 18:25:13 librenms logstash[18366]: [2018-05-02T18:25:13,075][FATAL][logstash.runner          ] An unexpected error occurred! {:error=>java.nio.file.AccessDeniedException: /usr/share/logstash/data/.loc
May 02 18:25:13 librenms logstash[18366]: logstash_minus_core.lib.logstash.runner.RUBY$method$run$0$__VARARGS__(usr/share/logstash/logstash_minus_core/lib/logstash//usr/share/logstash/logstash-core/lib/logst
May 02 18:25:13 librenms logstash[18366]: [2018-05-02T18:25:13,101][ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: org.jruby.exceptions.RaiseException: (SystemExit) exit

Here is the status output when it is running right before stopping

librenms@librenms:/etc/logstash$ sudo service logstash status
● logstash.service - logstash
Loaded: loaded (/etc/systemd/system/logstash.service; disabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/logstash.service.d
└─elastiflow.conf
Active: active (running) since Wed 2018-05-02 18:32:25 EDT; 18s ago
Main PID: 28510 (java)
Tasks: 17
Memory: 536.0M
CPU: 49.896s
CGroup: /system.slice/logstash.service
└─28510 /usr/bin/java -Xms1g -Xmx1g -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.awt.headless=true -Dfile.encoding=UTF-8

May 02 18:32:25 librenms systemd[1]: Started logstash.

2

What version, how did you install it?

3

Version 6.2.4 I installed it through apt-get.

4

I had to make a new directory and give it options 755 and own to logstash. Everything started working. The permissions and ownership on the original data file was the same. The only other change I made was that /var/log/logstash was owned by logstash:root and the files inside were owned by root. Changed all that to logstash:logstash as well. Service has been running for 4 minutes without stop. That is a success to me.

5

What OS>?

6

This is for Ubuntu Server 16.04

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.