Failed to start Elasticsearch | 'failed' state with result 'signal'

Elastic Stack Elasticsearch
1

Today I tried to install Elasticseatch on my new Ubuntu 22.04 server.

The setup did not show any error.

Nevertheless, at startup always comes this error:

-- Support: ***

-- The unit elasticsearch.service completed and consumed the indicated resources.
Jun 25 19:01:42 Zammad systemd[1]: Starting Elasticsearch...
-- Subject: A start job for unit elasticsearch.service has begun execution
-- Defined-By: systemd
-- Support: ***

-- A start job for unit elasticsearch.service has begun execution.

-- The job identifier is 253.
Jun 25 19:01:49 Zammad systemd[1]: elasticsearch.service: Main process exited, code=killed, status=9/KILL
-- Subject: Unit process exited
-- Defined-By: systemd
-- Support: ***

-- An ExecStart= process belonging to unit elasticsearch.service has exited.

-- The process' exit code is 'killed' and its exit status is 9.
Jun 25 19:01:49 Zammad systemd[1]: elasticsearch.service: Failed with result 'signal'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: ***

-- The unit elasticsearch.service has entered the 'failed' state with result 'signal'.
Jun 25 19:01:49 Zammad systemd[1]: Failed to start Elasticsearch.
-- Subject: A start job for unit elasticsearch.service has failed
-- Defined-By: systemd
-- Support: ***

-- A start job for unit elasticsearch.service has finished with a failure.

-- The job identifier is 253 and the job result is failed.
Jun 25 19:01:49 Zammad systemd[1]: elasticsearch.service: Consumed 13.096s CPU time.
-- Subject: Resources consumed by unit runtime
-- Defined-By: systemd
-- Support: ***

-- The unit elasticsearch.service completed and consumed the indicated resources.

If you need more information, please let me know. It might take some time since I'm not very good at Linux.

If I missed a thread that already covers this topic, please forgive me.

2

Welcome to the community Alex.

You should check ES log. Check here.

And please use the icon </> for formatting logs, code, etc.

3

I am Sorry, but thank you for the Tip

here is the newest log file

[2023-06-26T06:56:44.419+0000][447][gc,init] CardTable entry size: 512
[2023-06-26T06:56:44.419+0000][447][gc     ] Using G1
[2023-06-26T06:56:44.430+0000][447][gc,init] Version: 20.0.1+9-29 (release)
[2023-06-26T06:56:44.430+0000][447][gc,init] CPUs: 20 total, 2 available
[2023-06-26T06:56:44.430+0000][447][gc,init] Memory: 31797M
[2023-06-26T06:56:44.430+0000][447][gc,init] Large Page Support: Disabled
[2023-06-26T06:56:44.430+0000][447][gc,init] NUMA Support: Disabled
[2023-06-26T06:56:44.430+0000][447][gc,init] Compressed Oops: Enabled (Zero based)
[2023-06-26T06:56:44.430+0000][447][gc,init] Heap Region Size: 8M
[2023-06-26T06:56:44.430+0000][447][gc,init] Heap Min Capacity: 15904M
[2023-06-26T06:56:44.430+0000][447][gc,init] Heap Initial Capacity: 15904M
[2023-06-26T06:56:44.430+0000][447][gc,init] Heap Max Capacity: 15904M
[2023-06-26T06:56:44.430+0000][447][gc,init] Pre-touch: Disabled
[2023-06-26T06:56:44.430+0000][447][gc,init] Parallel Workers: 2
[2023-06-26T06:56:44.430+0000][447][gc,init] Concurrent Workers: 1
[2023-06-26T06:56:44.430+0000][447][gc,init] Concurrent Refinement Workers: 2
[2023-06-26T06:56:44.430+0000][447][gc,init] Periodic GC: Disabled
[2023-06-26T06:56:44.430+0000][447][gc,metaspace] CDS archive(s) not mapped
[2023-06-26T06:56:44.430+0000][447][gc,metaspace] Compressed class space mapped at: 0x00000007c0000000-0x0000000800000000, reserved size: 1073741824
[2023-06-26T06:56:44.430+0000][447][gc,metaspace] Narrow klass base: 0x0000000000000000, Narrow klass shift: 3, Narrow klass range: 0x800000000
[2023-06-26T06:56:44.519+0000][447][gc,heap,exit] Heap
[2023-06-26T06:56:44.519+0000][447][gc,heap,exit]  garbage-first heap   total 16285696K, used 16384K [0x00000003de000000, 0x00000007c0000000)
[2023-06-26T06:56:44.519+0000][447][gc,heap,exit]   region size 8192K, 2 young (16384K), 0 survivors (0K)
[2023-06-26T06:56:44.519+0000][447][gc,heap,exit]  Metaspace       used 3870K, committed 3904K, reserved 1114112K
[2023-06-26T06:56:44.519+0000][447][gc,heap,exit]   class space    used 302K, committed 320K, reserved 1048576K
4

That is gc.log, you should have .log, it's usually the largest file in the same dirctory.

5

I am sorry but there are no non gc files

root@Zammad:~# ls /var/log/elasticsearch
gc.log  gc.log.00  gc.log.01  gc.log.02  gc.log.03  gc.log.04  gc.log.05  gc.log.06  gc.log.07  gc.log.08  gc.log.09  gc.log.10  gc.log.11  gc.log.12  gc.log.13  gc.log.14  gc.log.15  gc.log.16  gc.log.17  gc.log.18  gc.log.19  gc.log.20
root@Zammad:~#
6

Have you setup cluster.name: <someclustername>?

7

No my Config is:

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
network.host: 192.168.178.66
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
#
# ---------------------------------- Security ----------------------------------
#
#                                 *** WARNING ***
#
# Elasticsearch security features are not enabled by default.
# These features are free, but require configuration changes to enable them.
# This means that users don ^`^yt have to provide credentials and can get full access
# to the cluster. Network connections are also not encrypted.
#
# To protect your data, we strongly encourage you to enable the Elasticsearch security features. 
# Refer to the following documentation for instructions.
#
# https://www.elastic.co/guide/en/elasticsearch/reference/7.16/configuring-stack-security.html

I tried to install it like the Documentation form DigitalOcean

8

Add in elasticsearch.yml:

cluster.name: something
node.name: something

Names are not checked, put whatever you like and restart ES.

Edit: If you have problem with access because limitation network.host: 192.168.178.66 put network.host: 0.0.0.0

9

melting_face

elasticsearch.service - Elasticsearch
     Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Mon 2023-06-26 08:57:00 UTC; 27s ago
       Docs: https://www.elastic.co
    Process: 912 ExecStart=/usr/share/elasticsearch/bin/systemd-entrypoint -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=1/FAILURE)
   Main PID: 912 (code=exited, status=1/FAILURE)
        CPU: 954ms

Jun 26 08:56:59 Zammad systemd[1]: Starting Elasticsearch...
Jun 26 08:57:00 Zammad systemd-entrypoint[1037]: encountered [2] errors parsing [/etc/elasticsearch/jvm.options]
Jun 26 08:57:00 Zammad systemd-entrypoint[1037]: [1]: encountered improperly formatted JVM option in [/etc/elasticsearch/jvm.options] on line number [31]: [ -Xms4g]
Jun 26 08:57:00 Zammad systemd-entrypoint[1037]: [2]: encountered improperly formatted JVM option in [/etc/elasticsearch/jvm.options] on line number [32]: [ -Xmx4g]
Jun 26 08:57:00 Zammad systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
Jun 26 08:57:00 Zammad systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
Jun 26 08:57:00 Zammad systemd[1]: Failed to start Elasticsearch.

Now i know why i always used Windows Server :upside_down_face:

## IMPORTANT: JVM heap size
################################################################
##
## The heap size is automatically configured by Elasticsearch
## based on the available memory in your system and the roles
## each node is configured to fulfill. If specifying heap is
## required, it should be done through a file in jvm.options.d,
## and the min and max should be set to the same value. For
## example, to set the heap to 4 GB, create a new file in the
## jvm.options.d directory containing these lines:
##
 -Xms4g
 -Xmx4g
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/7.17/heap-size.html
## for more information
##
10

jvm.options is not OK(remove spaces at the begging) and should be in /etc/elasticsearch/jvm.options.d/jvm.options

11

Now i have another error:upside_down_face:

x elasticsearch.service - Elasticsearch
     Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Mon 2023-06-26 09:12:03 UTC; 25s ago
       Docs: https://www.elastic.co
    Process: 771 ExecStart=/usr/share/elasticsearch/bin/systemd-entrypoint -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=78)
   Main PID: 771 (code=exited, status=78)
        CPU: 16.904s

Jun 26 09:11:54 Zammad systemd[1]: Starting Elasticsearch...
Jun 26 09:12:03 Zammad systemd-entrypoint[771]: ERROR: [1] bootstrap checks failed. You must address the points described in the following [1] lines before starting Elasticsearch.
Jun 26 09:12:03 Zammad systemd-entrypoint[771]: bootstrap check failure [1] of [1]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] mus>
Jun 26 09:12:03 Zammad systemd-entrypoint[771]: ERROR: Elasticsearch did not exit normally - check the logs at /var/log/elasticsearch/my-application.log
Jun 26 09:12:03 Zammad systemd[1]: elasticsearch.service: Main process exited, code=exited, status=78/CONFIG
Jun 26 09:12:03 Zammad systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
Jun 26 09:12:03 Zammad systemd[1]: Failed to start Elasticsearch.
Jun 26 09:12:03 Zammad systemd[1]: elasticsearch.service: Consumed 16.904s CPU time.
12

I added
discovery.seed_hosts: ["127.0.0.1", "0.0.0.0"]
now there is no error anymore

Thank you very much

13

No need for seed_hosts if is the single node. Usually this configuration in elasticsearch.yml is enough for the single node.

cluster.name: es
node.name: node1
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch

network.host: 0.0.0.0
http.port: 9200
discovery.type: single-node
14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.