Logstash stops running after running for few seconds

Elastic Stack Logstash
1

Hi,

My logstash stops running after 4 seconds of starting it, i get this output for systemctl status logstash -l

logstash.service - logstash
Loaded: loaded (/etc/systemd/system/logstash.service; disabled; vendor preset: disabled)
Active: active (running) since Thu 2017-08-03 18:46:53 UTC; 302ms ago
Main PID: 7521 (java)
CGroup: /system.slice/logstash.service
ââ7521 /usr/bin/java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -Djava.awt.headless=true -Dfile.encoding=UTF-8 -XX:+HeapDumpOnOutOfMemoryError -Xmx4g -Xms1g -Xss2048k -Djffi.boot.library.path=/usr/share/logstash/vendor/jruby/lib/jni -Xbootclasspath/a:/usr/share/logstash/vendor/jruby/lib/jruby.jar -classpath : -Djruby.home=/usr/share/logstash/vendor/jruby -Djruby.lib=/usr/share/logstash/vendor/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh org.jruby.Main /usr/share/logstash/lib/bootstrap/environment.rb logstash/runner.rb --path.settings /etc/logstash

Aug 03 18:46:53 ip-10-x-x-250.domain.com systemd[1]: Started logstash.
Aug 03 18:46:53 ip-10-x-x-250.domain.com systemd[1]: Starting logstash...

There is no log file created in /var/log/logstash.

Any help is highly appreciated.

2

Although when i do a test against the config file, it pases the test and also when i start logstash using the command

bin/logstash -f /path/to/config/file , it gives the desired output, but it doesnt work when i try to start it as systemctl start logstash

3

hope when you are running it as user you are using 'sudo'

4

I am a root on the server.

5

Can anybody help me in resolving this issue? It has been down for sometime now.

6

when you start it as service logstash start did you check if its running or not ?
start the logstash and show me the output of
$ ps aux | grep logstash

7

everytime i run this command, i can see a new PID which means that it is trying to restart logstash every few seconds and is failing, finally after few unsuccessful attempts it fails forever and never restarts untill i run the start command again.

8

did you check logstash logs in /var/log/logstash?

9

Nothing logged to related to the startup failure, but here is the log i see :slight_smile:
[2017-08-04T15:22:13,694][INFO ][logstash.pipeline ] Starting pipeline {"id"=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>250}
[2017-08-04T15:38:42,520][INFO ][logstash.pipeline ] Pipeline main started
[2017-08-04T15:38:42,531][WARN ][logstash.runner ] SIGHUP received.
[2017-08-04T15:38:42,545][WARN ][logstash.runner ] SIGTERM received. Shutting down the agent.
[2017-08-04T15:38:42,558][ERROR][logstash.inputs.metrics ] Failed to create monitoring event {:message=>"For path: http_address", :error=>"LogStash::Instrument::MetricStore::MetricNotFound"}
[2017-08-04T15:38:42,589][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2017-08-04T15:38:43,565][WARN ][logstash.agent ] stopping pipeline {:id=>".monitoring-logstash"}
[2017-08-04T15:38:45,517][WARN ][logstash.agent ] stopping pipeline {:id=>"main"}

10

run it using strace and save the output to a file and share it with us.

11

whats the command to run it using strace?

12

before that did you try to run it manually from /usr/share/logstash ?
$ cd /usr/share/logstash
$ bin/logstash -f /etc/logstash.d/conf/yourfile.conf

and see if it runs properly

13

Yes i ran it manually using this command, and i am still running it using just one config file. It works like a champ. But not with systemctl start logstash

14

ok so start it as service again and show me the output of
$systemctl status logstash

15

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.