Logstash 7.x service fails to start on Ubuntu 18.04 LTS with OpenJDK 11 JDK as a service but starts otherwise

I am trying to use logstash a SystemV service on Ubuntu 18.04 LTS. Whenever I try to use the service command, the service startup fails

> service logstash start && journalctl -u logstash
logstash.service: Failed to execute command: Exec format error
logstash.service: Failed at step EXEC spawning /usr/share/logstash/bin/logstash: Exec format error

I installed openjdk-11-jdk on my server for my application so I suppose this is what logstash uses to start

I am trying to use logstash a SystemV service on Ubuntu 18.04 LTS. Whenever I try to use the service command, the service startup fails

> service logstash start && journalctl -u logstash
logstash.service: Failed to execute command: Exec format error
logstash.service: Failed at step EXEC spawning /usr/share/logstash/bin/logstash: Exec format error

I installed openjdk-11-jdk on my server for my application so I suppose this is what logstash uses to start

> which java

/usr/bin/java

> /usr/bin/java --version

openjdk 11.0.8 2020-07-14
OpenJDK Runtime Environment (build 11.0.8+10-post-Ubuntu-0ubuntu118.04.1)
OpenJDK 64-Bit Server VM (build 11.0.8+10-post-Ubuntu-0ubuntu118.04.1, mixed mode, sharing)

I installed logstash using APT from : deb https://artifacts.elastic.co/packages/7.x/apt stable main

It seems that such errors occur when Java compatibility issues but from Elastic documentation OpenJDK 11 and logstash on Ubuntu 18.04 LTS should work.

The /etc/systemd/system/logstash.service file is :

[Unit]
Description=logstash

[Service]
Type=simple
User=logstash
Group=logstash
# Load env vars from /etc/default/ and /etc/sysconfig/ if they exist.
# Prefixing the path with '-' makes it try to load, but if the file doesn't
# exist, it continues onward.
EnvironmentFile=-/etc/default/logstash
EnvironmentFile=-/etc/sysconfig/logstash
ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash"
Restart=always
WorkingDirectory=/
Nice=19
LimitNOFILE=16384

# When stopping, how long to wait before giving up and sending SIGKILL?
# Keep in mind that SIGKILL on a process can cause data loss.
TimeoutStopSec=infinity

[Install]
WantedBy=multi-user.target

Using the following command :

/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash"

Seems to work :

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jruby.ext.openssl.SecurityHelper (file:/tmp/jruby-60735/jruby15836533716820282820jopenssl.jar) to field java.security.MessageDigest.provider
WARNING: Please consider reporting this to the maintainers of org.jruby.ext.openssl.SecurityHelper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties
[2020-08-26T17:35:16,236][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.9.0", "jruby.version"=>"jruby 9.2.12.0 (2.5.7) 2020-07-01 db01a49ba6 OpenJDK 64-Bit Server VM 11.0.8+10-post-Ubuntu-0ubuntu118.04.1 on 11.0.8+10-post-Ubuntu-0ubuntu118.04.1 +indy +jit [linux-x86_64]"}
....
[2020-08-26T17:35:24,858][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2020-08-26T17:35:25,057][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

What can be the issue ?

Can you check the Logstash logs, not just the response from service logstash start, and see what it mentions?

From what I see logs remain empty when I attempt to start Logstash using the service command :

/var/log/logstash# ll

total 8
drwxrwxr-x  2 logstash root   4096 Aug 26 16:55 ./
drwxrwxr-x 12 root     syslog 4096 Aug 27 00:00 ../
-rw-r--r--  1 root     root      0 Aug 26 16:55 logstash-deprecation.log
-rw-r--r--  1 root     root      0 Aug 27 07:41 logstash-plain.log
-rw-r--r--  1 root     root      0 Aug 26 16:55 logstash-slowlog-plain.log

The logstash-plain.log only fills when I start Logstash without the service command

Ok, it seems that it was an issue with a custom line added in the /usr/share/logstash/bin/logstash file, I will confirm this and close this thread once done

Ok, I confirm that using the default configuration file /usr/share/logstash/bin/logstash works fine for me.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.