Can't run Logstash as a Service

Elastic Stack Logstash
1

If i run logstash manually it works fine and sends all data in the right way.

But when i try to run it as a service it shows this error:

Errore logstash as a service
Errore logstash as a service1447×300 25.6 KB

Cmd that i use: sudo systemctl start logstash.service
OS Ubuntu 20.04.1
Logstash version: 7.9.2

Pls help me, i need tofix it quickly

2

You need to show us the logstash logs, possibly in /var/log/logstash.

3

It doesn't generate any new log :frowning:

4

You need to share your system logs as well.

Look at /var/log/messages or /var/log/syslog when you tried to start the logstash service to see why it was not able to start.

5

It doesn't generate any new log

6

Should be something in /var/log/messages for Red Hat based distributions or /var/log/syslog for Debian based distributions.

Try to run sudo systemctl start logstash and check one of those files, without seeings logs of the system is not possible to know what is happening.

Also check the permissions to the /var/log/logstash folder to see if the logstash user has permissions to write on this folder.

7

image
image1237×154 89.9 KB

8

There is your error, logstash could not find java.

You will need to install java or set JAVA_HOME for the logstash processs.

You could also update to a newer version, starting with version 7.10 logstash now has a bundled java, but version 7.9 does not, so you need a java installation.

9

i already have java,in:
java: /usr/bin/java /usr/share/java /usr/lib/jvm/java-11-oracle/bin/java

Where i set JAVA_HOME?

10

What the following command returns?

sudo -u logstash which java

This will tell you if the logstash user, the one that runs the service, can find the java binary in your system.

You could also add the variable into the file /etc/default/logstash or /etc/sysconfig/logstash depending on the distribution of your system.

You would need to add a line like this:

JAVA_HOME=/path/to/java/home
11

If i lunch the command it doesn't give something in return

I use Ubuntu 20.04.1, so i need to modify /etc/default/logstash, right?

12

If the command does not return anything, than the logstash user can not find java in your system, that's the reason you cannot start the service.

You could add the variable to the /etc/default/logstash, but it would be better if you just upgrade the logstash to a new version as it already has a bundled jdk.

13

I tried to start and nothing arrivied in Kibana, i looked at syslog and i found this:

image
image1486×61 52 KB

image
image1870×306 362 KB

image
image1797×309 359 KB

image
image1845×91 92.6 KB

14

Please, do not share screenshot of log messages, just copy the plain text and share it using the preformatted text button </>.

It is really hard to see error logs on screenshots and is not possible to search on them.

Your error is pretty clear, the logstash user does not have permission to write in the directory /var/log/logstash, so it can't create the log file.

Change the permission to the directory

sudo chown logstash:logstash /var/log/logstash
15

i have the same error after using the command sudo chown

main ERROR RollingFileManager (/var/log/logstash/logstash-plain.log) java.io.FileNotFoundException: /var/log/logstash/logstash-plain.log (Permission denied) java.io.FileNotFoundException: /var/log/logstash/logstash-plain.log (Permission denied)
16

Did you tried to run logstash before as the root user? If you did it would create the file with the root user as the owner.

Just remove the file and start the service again.

sudo rm /var/log/logstash/logstash-plain.log
17

I only run logstash manually before, as root

Now i have this:

[ERROR][org.logstash.execution.AbstractPipelineExt] Logstash failed to create queue.

[ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaLang::IllegalStateException", :message=>"java.nio.file.AccessDeniedException: /usr/share/logstash/queue_data/main/.lock", :backtrace=>["org.logstash.execution.AbstractPipelineExt.openQueue(AbstractPipelineExt.java:204)", "org.logstash.execution.AbstractPipelineExt$INVOKER$i$0$0$openQueue.call(AbstractPipelineExt$INVOKER$i$0$0$openQueue.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:831)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$initialize$0(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:45)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:86)", "org.jruby.RubyClass.newInstance(RubyClass.java:939)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0(/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0$__VARARGS__(/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "usr.share.logstash.logstash_minus_core.lib.logstash.agent.RUBY$block$converge_state$2(/usr/share/logstash/logstash-core/lib/logstash/agent.rb:357)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52)", "org.jruby.runtime.Block.call(Block.java:139)", "org.jruby.RubyProc.call(RubyProc.java:318)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)", "java.base/java.lang.Thread.run(Thread.java:834)"]}
Mar 10 15:42:41 lnx01bigdata logstash[140667]: warning: thread "Converge PipelineAction::Create<main>" terminated with exception (report_on_exception is true):
18

That's the issue, when you run it as the root user, it creates a couple of files owned by the root user, mainly the log file and some files if you are using persistent queues, which seems your issue now.

You will need to remove or change the permission of those files created when you run it as a root, check the log, see what file it says that it does not have permission and delete or change the permissions to the logstash user.

19

could be this?

java.nio.file.AccessDeniedException: /usr/share/logstash/queue_data/main/.lock
20

Yes, check the description of the error AccessDeniedException, this means that logstash could not access this file, so you need to change it permissions to the logstash user.