Beginner: Trouble getting started

Deployed elasticsearch to an Ubuntu VM within Azure cloud. Edited the elasticsearch.yaml for the port, cluster.initial_master_nodes, and the network host:

[network.host](https://network.host/): "localhost"
http.port: 9200
cluster.initial_master_nodes: ["PRIVATE IP"]

Then edited the jvm.options to set the heap size:

-Xms128m
-Xmx128m

When starting to run the service it failed and below is the output of the journalctl -xe

root@kai-elk:/etc/elasticsearch# journalctl -xe

Aug 26 22:17:02 kai-elk systemd-entrypoint[24922]: ... 10 more

Aug 26 22:17:02 kai-elk systemd-entrypoint[24922]: Caused by: while scanning a simple key

Aug 26 22:17:02 kai-elk systemd-entrypoint[24922]: in 'reader', line 57, column 1:

Aug 26 22:17:02 kai-elk systemd-entrypoint[24922]: http.port:9200

Aug 26 22:17:02 kai-elk systemd-entrypoint[24922]: ^

Aug 26 22:17:02 kai-elk systemd-entrypoint[24922]: could not find expected ':'

Aug 26 22:17:02 kai-elk systemd-entrypoint[24922]: in 'reader', line 58, column 1:

Aug 26 22:17:02 kai-elk systemd-entrypoint[24922]: cluster.initial_master_nodes: [" ...

Aug 26 22:17:02 kai-elk systemd-entrypoint[24922]: ^

Aug 26 22:17:02 kai-elk systemd-entrypoint[24922]: at org.yaml.snakeyaml.scanner.ScannerImpl.stalePossibleSimpleKeys([ScannerImpl.java:464](https://scannerimpl.java:464/))

Aug 26 22:17:02 kai-elk systemd-entrypoint[24922]: at org.yaml.snakeyaml.scanner.ScannerImpl.needMoreTokens([ScannerImpl.java:278](https://scannerimpl.java:278/))

Aug 26 22:17:02 kai-elk systemd-entrypoint[24922]: at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken([ScannerImpl.java:226](https://scannerimpl.java:226/))

Aug 26 22:17:02 kai-elk systemd-entrypoint[24922]: at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingKey.produce([ParserImpl.java:558](https://parserimpl.java:558/))

Aug 26 22:17:02 kai-elk systemd-entrypoint[24922]: at org.yaml.snakeyaml.parser.ParserImpl.peekEvent([ParserImpl.java:158](https://parserimpl.java:158/))

Aug 26 22:17:02 kai-elk systemd-entrypoint[24922]: at org.yaml.snakeyaml.parser.ParserImpl.getEvent([ParserImpl.java:168](https://parserimpl.java:168/))

Aug 26 22:17:02 kai-elk systemd-entrypoint[24922]: at com.fasterxml.jackson.dataformat.yaml.YAMLParser.nextToken([YAMLParser.java:355](https://yamlparser.java:355/))

Aug 26 22:17:02 kai-elk systemd-entrypoint[24922]: ... 15 more

Aug 26 22:17:02 kai-elk systemd-entrypoint[24978]: encountered [2] errors parsing [/etc/elasticsearch/jvm.options]

Aug 26 22:17:02 kai-elk systemd-entrypoint[24978]: [1]: encountered improperly formatted JVM option in [/etc/elasticsearch/jvm.options] on line number [31]: [ -Xms128m]

Aug 26 22:17:02 kai-elk systemd-entrypoint[24978]: [2]: encountered improperly formatted JVM option in [/etc/elasticsearch/jvm.options] on line number [32]: [ -Xmx128m]

Aug 26 22:17:02 kai-elk systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE

-- Subject: Unit process exited

-- Defined-By: systemd

-- Support: http://www.ubuntu.com/support

--

-- An ExecStart= process belonging to unit elasticsearch.service has exited.

--

-- The process' exit code is 'exited' and its exit status is 1.

Aug 26 22:17:02 kai-elk sudo[24861]: pam_unix(sudo:session): session closed for user root

Aug 26 22:17:02 kai-elk systemd[1]: elasticsearch.service: Failed with result 'exit-code'.

-- Subject: Unit failed

-- Defined-By: systemd

-- Support: http://www.ubuntu.com/support

--

-- The unit elasticsearch.service has entered the 'failed' state with result 'exit-code'.

Aug 26 22:17:02 kai-elk systemd[1]: Failed to start Elasticsearch.

-- Subject: A start job for unit elasticsearch.service has failed

-- Defined-By: systemd

-- Support: http://www.ubuntu.com/support

--

-- A start job for unit elasticsearch.service has finished with a failure.

--

-- The job identifier is 2340 and the job result is failed.

Aug 26 22:18:51 kai-elk sudo[25014]: root : TTY=pts/0 ; PWD=/etc/elasticsearch ; USER=root ; COMMAND=/usr/sbin/service elasticsearch start

Aug 26 22:18:51 kai-elk sudo[25014]: pam_unix(sudo:session): session opened for user root by azureuser(uid=0)

Aug 26 22:18:52 kai-elk systemd[1]: Starting Elasticsearch...

-- Subject: A start job for unit elasticsearch.service has begun execution

-- Defined-By: systemd

-- Support: http://www.ubuntu.com/support

--

-- A start job for unit elasticsearch.service has begun execution.

--

-- The job identifier is 2437.

Aug 26 22:18:54 kai-elk systemd-entrypoint[25128]: encountered [2] errors parsing [/etc/elasticsearch/jvm.options]

Aug 26 22:18:54 kai-elk systemd-entrypoint[25128]: [1]: encountered improperly formatted JVM option in [/etc/elasticsearch/jvm.options] on line number [31]: [ -Xms128m]

Aug 26 22:18:54 kai-elk systemd-entrypoint[25128]: [2]: encountered improperly formatted JVM option in [/etc/elasticsearch/jvm.options] on line number [32]: [ -Xmx128m]

Aug 26 22:18:54 kai-elk systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE

-- Subject: Unit process exited

-- Defined-By: systemd

-- Support: http://www.ubuntu.com/support

--

-- An ExecStart= process belonging to unit elasticsearch.service has exited.

--

-- The process' exit code is 'exited' and its exit status is 1.

Aug 26 22:18:54 kai-elk sudo[25014]: pam_unix(sudo:session): session closed for user root

Aug 26 22:18:54 kai-elk systemd[1]: elasticsearch.service: Failed with result 'exit-code'.

-- Subject: Unit failed

-- Defined-By: systemd

-- Support: http://www.ubuntu.com/support

--

-- The unit elasticsearch.service has entered the 'failed' state with result 'exit-code'.

Aug 26 22:18:54 kai-elk systemd[1]: Failed to start Elasticsearch.

-- Subject: A start job for unit elasticsearch.service has failed

-- Defined-By: systemd

-- Support: http://www.ubuntu.com/support

--

-- A start job for unit elasticsearch.service has finished with a failure.

--

-- The job identifier is 2437 and the job result is failed.

Aug 26 22:20:21 kai-elk sshd[25163]: error: kex_exchange_identification: Connection closed by remote host

Aug 26 22:21:26 kai-elk sshd[25177]: error: kex_exchange_identification: Connection closed by remote host

Aug 26 22:22:16 kai-elk sshd[25196]: error: kex_exchange_identification: Connection closed by remote host

Aug 26 22:22:27 kai-elk sshd[25200]: Unable to negotiate with [141.98.10.250](https://141.98.10.250/) port 36086: no matching key exchange method found. Their offer: diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1 [preauth]

Aug 26 22:28:13 kai-elk python3[709]: 2021-08-26T22:28:13.546163Z INFO ExtHandler ExtHandler [HEARTBEAT] Agent WALinuxAgent-2.4.0.2 is running as the goal state agent [DEBUG HeartbeatCounter: 3;HeartbeatId: B6BE69CB-3F8F-40DC-BE79-B8E72971364D;DroppedPackets: 0;UpdateGSErrors: 0;AutoUpdate: 1]

lines 1221-1304/1304 (END)

Can someone help me determine what I did wrong?

Hello,

Can you show us your /etc/elasticsearch/jvm.options file?

@jmsbooth Welcome to the community

Also your elasticsearch.yml

The errors indicate a syntax error.

Please format your code with the </> buttons

I do not think I have ever seen Elasticsearch run with such a small heap so I would recommend increasing it to at least 512MB.

2 Likes

jvm.options

################################################################

## GC configuration
8-13:-XX:+UseConcMarkSweepGC
8-13:-XX:CMSInitiatingOccupancyFraction=75
8-13:-XX:+UseCMSInitiatingOccupancyOnly

## G1GC Configuration
# NOTE: G1 GC is only supported on JDK version 10 or later
# to use G1GC, uncomment the next two lines and update the version on the
# following three lines to your version of the JDK
# 10-13:-XX:-UseConcMarkSweepGC
# 10-13:-XX:-UseCMSInitiatingOccupancyOnly
14-:-XX:+UseG1GC

## JVM temporary directory
-Djava.io.tmpdir=${ES_TMPDIR}

## heap dumps

# generate a heap dump when an allocation from the Java heap fails; heap dumps
# are created in the working directory of the JVM unless an alternative path is
# specified
-XX:+HeapDumpOnOutOfMemoryError

# specify an alternative path for heap dumps; ensure the directory exists and
# has sufficient space
-XX:HeapDumpPath=/var/lib/elasticsearch

# specify an alternative path for JVM fatal error logs
-XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log

## JDK 8 GC logging
8:-XX:+PrintGCDetails
8:-XX:+PrintGCDateStamps
8:-XX:+PrintTenuringDistribution
8:-XX:+PrintGCApplicationStoppedTime
8:-Xloggc:/var/log/elasticsearch/gc.log
8:-XX:+UseGCLogFileRotation
8:-XX:NumberOfGCLogFiles=32
8:-XX:GCLogFileSize=64m

# JDK 9+ GC logging
9-:-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m
9-:-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m

elasticsearch.yml

#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
network.host: "localhost"
http.port: 9200
cluster.initial_master_nodes: ["PRIVATEIP"]
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true

I figured and plan to expand it. Its a lab for me so tried going small to begin with.

That is too small IMHO. Elasticsearch does not scale down that far.

Hello,

Part of your jvm.options file is missing.

 line number [31]:  &  line number [32]:

I have to agree with @Christian_Dahlqvist that is very small for heap.