Elasticsearch getting failed to start the service

Elastic Stack Elasticsearch
1

image
image830×722 15 KB

Below i mentioned error log for elasticsearch.
I don't know how to resolve this, Please anybody help me to resolve this ASAP. Because i want to start this in development server.

<The job identifier is 14563 and the job result is failed.
Mar 14 15:28:36 los-master sudo[63019]: pam_unix(sudo:session): session closed for user root
Mar 14 15:28:50 los-master kernel: [UFW BLOCK] IN=eth0 OUT= MAC=96:00:01:fe:f7:24:d2:74:7f:6e:37:e3:08:00 SRC=5.188.87.6 DST=65.109.236.130 LEN=40 TOS=0x00 PREC=0x00 TTL=246 ID=19155 PROTO=TCP SPT=56176 DPT=1208 WINDOW=1024 RES=0x00 SYN>
Mar 14 15:29:02 los-master filebeat[61640]: 2023-03-14T15:29:02.600+0530 INFO [monitoring] log/log.go:184 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cgroup":{"cpuacct":{"tota>
Mar 14 15:29:02 los-master filebeat[61640]: 2023-03-14T15:29:02.735+0530 ERROR [publisher_pipeline_output] pipeline/output.go:154 Failed to connect to backoff(async(tcp://localhost:5044)): dial tcp [::1]:5044>
Mar 14 15:29:02 los-master filebeat[61640]: 2023-03-14T15:29:02.739+0530 INFO [publisher] pipeline/retry.go:219 retryer: send unwait signal to consumer
Mar 14 15:29:02 los-master filebeat[61640]: 2023-03-14T15:29:02.739+0530 INFO [publisher] pipeline/retry.go:223 done
Mar 14 15:29:02 los-master filebeat[61640]: 2023-03-14T15:29:02.739+0530 INFO [publisher_pipeline_output] pipeline/output.go:145 Attempting to reconnect to backoff(async(tcp://localhost:5044)) with 17 reconne>
Mar 14 15:29:11 los-master kernel: [UFW BLOCK] IN=eth0 OUT= MAC=96:00:01:fe:f7:24:d2:74:7f:6e:37:e3:08:00 SRC=196.188.75.247 DST=65.109.236.130 LEN=40 TOS=0x00 PREC=0x00 TTL=53 ID=14465 PROTO=TCP SPT=6404 DPT=23 WINDOW=12585 RES=0x00 SY>
Mar 14 15:29:14 los-master sudo[63223]: root : TTY=pts/1 ; PWD=/root ; USER=root ; COMMAND=/usr/bin/nano /etc/elasticsearch/jvm.options
Mar 14 15:29:14 los-master sudo[63223]: pam_unix(sudo:session): session opened for user root by root(uid=0)
Mar 14 15:29:32 los-master filebeat[61640]: 2023-03-14T15:29:32.570+0530 INFO [add_docker_metadata] docker/watcher.go:309 No events received within 10m0s, restarting watch call
Mar 14 15:29:32 los-master filebeat[61640]: 2023-03-14T15:29:32.591+0530 INFO [monitoring] log/log.go:184 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cgroup":{"cpuacct":{"tota>
Mar 14 15:29:34 los-master sudo[63223]: pam_unix(sudo:session): session closed for user root
Mar 14 15:29:36 los-master kernel: [UFW BLOCK] IN=eth0 OUT= MAC=96:00:01:fe:f7:24:d2:74:7f:6e:37:e3:08:00 SRC=94.102.61.49 DST=65.109.236.130 LEN=44 TOS=0x00 PREC=0x00 TTL=248 ID=54321 PROTO=TCP SPT=50172 DPT=5684 WINDOW=65535 RES=0x00 >
Mar 14 15:29:38 los-master filebeat[61640]: 2023-03-14T15:29:38.475+0530 ERROR [publisher_pipeline_output] pipeline/output.go:154 Failed to connect to backoff(async(tcp://localhost:5044)): dial tcp 127.0.0.1:>
Mar 14 15:29:38 los-master filebeat[61640]: 2023-03-14T15:29:38.475+0530 INFO [publisher_pipeline_output] pipeline/output.go:145 Attempting to reconnect to backoff(async(tcp://localhost:5044)) with 18 reconne>
Mar 14 15:29:38 los-master filebeat[61640]: 2023-03-14T15:29:38.475+0530 INFO [publisher] pipeline/retry.go:219 retryer: send unwait signal to consumer
Mar 14 15:29:38 los-master filebeat[61640]: 2023-03-14T15:29:38.475+0530 INFO [publisher] pipeline/retry.go:223 done
Mar 14 15:29:47 los-master sudo[63225]: root : TTY=pts/1 ; PWD=/root ; USER=root ; COMMAND=/usr/bin/systemctl daemon-reload
Mar 14 15:29:47 los-master sudo[63225]: pam_unix(sudo:session): session opened for user root by root(uid=0)
Mar 14 15:29:47 los-master systemd[1]: Reloading.
Mar 14 15:29:48 los-master sudo[63225]: pam_unix(sudo:session): session closed for user root
Mar 14 15:29:52 los-master sudo[63261]: root : TTY=pts/1 ; PWD=/root ; USER=root ; COMMAND=/usr/bin/systemctl start elasticsearch
Mar 14 15:29:52 los-master sudo[63261]: pam_unix(sudo:session): session opened for user root by root(uid=0)
Mar 14 15:29:52 los-master systemd[1]: Starting Elasticsearch...
-- Subject: A start job for unit elasticsearch.service has begun execution
-- Defined-By: systemd
-- Support: Enterprise open source support | Ubuntu

-- A start job for unit elasticsearch.service has begun execution.

-- The job identifier is 14648.
Mar 14 15:29:55 los-master systemd-entrypoint[63413]: encountered [2] errors parsing [/etc/elasticsearch/jvm.options]
Mar 14 15:29:55 los-master systemd-entrypoint[63413]: [1]: encountered improperly formatted JVM option in [/etc/elasticsearch/jvm.options] on line number [31]: [ -Xms512m]
Mar 14 15:29:55 los-master systemd-entrypoint[63413]: [2]: encountered improperly formatted JVM option in [/etc/elasticsearch/jvm.options] on line number [32]: [ -Xmx512m]
Mar 14 15:29:55 los-master systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
-- Subject: Unit process exited
-- Defined-By: systemd
-- Support: Enterprise open source support | Ubuntu

-- An ExecStart= process belonging to unit elasticsearch.service has exited.

-- The process' exit code is 'exited' and its exit status is 1.
Mar 14 15:29:55 los-master systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: Enterprise open source support | Ubuntu

-- The unit elasticsearch.service has entered the 'failed' state with result 'exit-code'.
Mar 14 15:29:55 los-master systemd[1]: Failed to start Elasticsearch.
-- Subject: A start job for unit elasticsearch.service has failed
-- Defined-By: systemd
-- Support: Enterprise open source support | Ubuntu

-- A start job for unit elasticsearch.service has finished with a failure.

-- The job identifier is 14648 and the job result is failed.
Mar 14 15:29:55 los-master sudo[63261]: pam_unix(sudo:session): session closed for user root>

2

From the error message it seems that /etc/elasticsearch/jvm.options
has an error (or missing) lines 31 and 32

3

please have a look at your logstash piplines (in/out)

4 
Mar 15 09:07:11 los-master systemd[1]: Starting Elasticsearch...
Mar 15 09:07:14 los-master systemd-entrypoint[72262]: encountered [2] errors parsing [/etc/elasticsearch/jvm.options]
Mar 15 09:07:14 los-master systemd-entrypoint[72262]: [1]: encountered improperly formatted JVM option in [/etc/elasticsearch/jvm.options] on line number [31]: [ -Xms512m]
Mar 15 09:07:14 los-master systemd-entrypoint[72262]: [2]: encountered improperly formatted JVM option in [/etc/elasticsearch/jvm.options] on line number [32]: [ -Xmx512m]
Mar 15 09:07:14 los-master systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
Mar 15 09:07:14 los-master systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
Mar 15 09:07:14 los-master systemd[1]: Failed to start Elasticsearch.
Mar 15 09:09:10 los-master systemd[1]: Starting Elasticsearch...
Mar 15 09:09:12 los-master systemd-entrypoint[72609]: encountered [2] errors parsing [/etc/elasticsearch/jvm.options]
Mar 15 09:09:12 los-master systemd-entrypoint[72609]: [1]: encountered improperly formatted JVM option in [/etc/elasticsearch/jvm.options] on line number [31]: [ -Xms512m]
Mar 15 09:09:12 los-master systemd-entrypoint[72609]: [2]: encountered improperly formatted JVM option in [/etc/elasticsearch/jvm.options] on line number [32]: [ -Xmx512m]
Mar 15 09:09:12 los-master systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
Mar 15 09:09:12 los-master systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
Mar 15 09:09:12 los-master systemd[1]: Failed to start Elasticsearch.
Mar 15 09:15:16 los-master systemd[1]: Starting Elasticsearch...
Mar 15 09:15:18 los-master systemd-entrypoint[72805]: encountered [2] errors parsing [/etc/elasticsearch/jvm.options]
Mar 15 09:15:18 los-master systemd-entrypoint[72805]: [1]: encountered improperly formatted JVM option in [/etc/elasticsearch/jvm.options] on line number [31]: [ -Xms512m]
Mar 15 09:15:18 los-master systemd-entrypoint[72805]: [2]: encountered improperly formatted JVM option in [/etc/elasticsearch/jvm.options] on line number [32]: [ -Xmx512m]
Mar 15 09:15:18 los-master systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
Mar 15 09:15:18 los-master systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
Mar 15 09:15:18 los-master systemd[1]: Failed to start Elasticsearch.
Mar 15 09:17:25 los-master systemd[1]: Starting Elasticsearch...
Mar 15 09:17:29 los-master systemd[1]: elasticsearch.service: Main process exited, code=killed, status=9/KILL
Mar 15 09:17:29 los-master systemd[1]: elasticsearch.service: Failed with result 'signal'.
Mar 15 09:17:29 los-master systemd[1]: Failed to start Elasticsearch.
Mar 15 09:23:12 los-master systemd[1]: Starting Elasticsearch...
Mar 15 09:23:15 los-master systemd-entrypoint[73385]: encountered [2] errors parsing [/etc/elasticsearch/jvm.options]
Mar 15 09:23:15 los-master systemd-entrypoint[73385]: [1]: encountered improperly formatted JVM option in [/etc/elasticsearch/jvm.options] on line number [31]: [ -Xms2g]
Mar 15 09:23:15 los-master systemd-entrypoint[73385]: [2]: encountered improperly formatted JVM option in [/etc/elasticsearch/jvm.options] on line number [32]: [ -Xmx2g]
Mar 15 09:23:15 los-master systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
Mar 15 09:23:15 los-master systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
Mar 15 09:23:15 los-master systemd[1]: Failed to start Elasticsearch.
Mar 15 09:32:20 los-master systemd[1]: Starting Elasticsearch...
Mar 15 09:32:22 los-master systemd-entrypoint[73569]: encountered [2] errors parsing [/etc/elasticsearch/jvm.options]
Mar 15 09:32:22 los-master systemd-entrypoint[73569]: [1]: encountered improperly formatted JVM option in [/etc/elasticsearch/jvm.options] on line number [31]: [ -Xms2g]
Mar 15 09:32:22 los-master systemd-entrypoint[73569]: [2]: encountered improperly formatted JVM option in [/etc/elasticsearch/jvm.options] on line number [32]: [ -Xmx2g]
Mar 15 09:32:22 los-master systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
Mar 15 09:32:22 los-master systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
Mar 15 09:32:22 los-master systemd[1]: Failed to start Elasticsearch.
Mar 15 09:39:56 los-master systemd[1]: Starting Elasticsearch...
Mar 15 09:39:58 los-master systemd-entrypoint[73990]: encountered [2] errors parsing [/etc/elasticsearch/jvm.options]
Mar 15 09:39:58 los-master systemd-entrypoint[73990]: [1]: encountered improperly formatted JVM option in [/etc/elasticsearch/jvm.options] on line number [31]: [ -Xms128m]
Mar 15 09:39:58 los-master systemd-entrypoint[73990]: [2]: encountered improperly formatted JVM option in [/etc/elasticsearch/jvm.options] on line number [32]: [ -Xmx128m]
Mar 15 09:39:58 los-master systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
Mar 15 09:39:58 los-master systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
Mar 15 09:39:58 los-master systemd[1]: Failed to start Elasticsearch.
5 
################################################################
## IMPORTANT: JVM heap size
################################################################
##
## The heap size is automatically configured by Elasticsearch
## based on the available memory in your system and the roles
## each node is configured to fulfill. If specifying heap is
## required, it should be done through a file in jvm.options.d,
## and the min and max should be set to the same value. For
## example, to set the heap to 4 GB, create a new file in the
## jvm.options.d directory containing these lines:
##
 -Xms128m
 -Xmx128m
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/7.17/heap-size.html
## for more information
##
################################################################


################################################################
## Expert settings
################################################################
##
## All settings below here are considered expert settings. Do
## not adjust them unless you understand what you are doing. Do
## not edit them in this file; instead, create a new file in the
## jvm.options.d directory containing your adjustments.
##
################################################################

## GC configuration
8-13:-XX:+UseConcMarkSweepGC
8-13:-XX:CMSInitiatingOccupancyFraction=75
8-13:-XX:+UseCMSInitiatingOccupancyOnly

## G1GC Configuration
# NOTE: G1 GC is only supported on JDK version 10 or later
# to use G1GC, uncomment the next two lines and update the version on the
# following three lines to your version of the JDK
# 10-13:-XX:-UseConcMarkSweepGC
# 10-13:-XX:-UseCMSInitiatingOccupancyOnly
14-:-XX:+UseG1GC

## JVM temporary directory
-Djava.io.tmpdir=${ES_TMPDIR}

## heap dumps

# generate a heap dump when an allocation from the Java heap fails; heap dumps
# are created in the working directory of the JVM unless an alternative path is
# specified
-XX:+HeapDumpOnOutOfMemoryError

# exit right after heap dump on out of memory error. Recommended to also use

There is any mistake on this

6

why configuring it in mbs
I would suggest to put in gbs.

for example :slight_smile:

-Xms4g

-Xmx4g

7

I do not think I have ever seen Elasticsearch run with a heap that small. I would recommend setting it to at least 512mb and esnore Elasticsearch have some access to 1GB of RAM in total. Even 512mb is considered a small heap though...

1 Like
8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.