New install - unable to start Elasticsearch

[rcoote (Rob Coote)]

Hello,

Fresh install on Ubuntu server following along with Rob's video. When starting Elasticsearch for the first time it fails.

admin@oeg-elk-01:~ sudo systemctl start elasticsearch Job for elasticsearch.service failed because the control process exited with error code. See "systemctl status elasticsearch.service" and "journalctl -xe" for details. admin@oeg-elk-01:~ sudo systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/elasticsearch.service.d
└─elasticsearch.conf
Active: failed (Result: exit-code) since Wed 2020-09-16 18:24:39 UTC; 10s ago
Docs: https://www.elastic.co
Process: 2806 ExecStart=/usr/share/elasticsearch/bin/systemd-entrypoint -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=1/FAILURE)
Main PID: 2806 (code=exited, status=1/FAILURE)

Sep 16 18:24:39 oeg-elk-01 systemd-entrypoint[2806]: at com.fasterxml.jackson.dataformat.yaml.YAMLParser.nextToken(YAMLParser.java:399)
Sep 16 18:24:39 oeg-elk-01 systemd-entrypoint[2806]: at org.elasticsearch.common.xcontent.json.JsonXContentParser.nextToken(JsonXContentParser.java:52)
Sep 16 18:24:39 oeg-elk-01 systemd-entrypoint[2806]: at org.elasticsearch.common.settings.Settings.fromXContent(Settings.java:645)
Sep 16 18:24:39 oeg-elk-01 systemd-entrypoint[2806]: at org.elasticsearch.common.settings.Settings.fromXContent(Settings.java:620)
Sep 16 18:24:39 oeg-elk-01 systemd-entrypoint[2806]: at org.elasticsearch.common.settings.Settings.access$400(Settings.java:82)
Sep 16 18:24:39 oeg-elk-01 systemd-entrypoint[2806]: at org.elasticsearch.common.settings.Settings$Builder.loadFromStream(Settings.java:1096)
Sep 16 18:24:39 oeg-elk-01 systemd-entrypoint[2806]: ... 9 more
Sep 16 18:24:39 oeg-elk-01 systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
Sep 16 18:24:39 oeg-elk-01 systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
Sep 16 18:24:39 oeg-elk-01 systemd[1]: Failed to start Elasticsearch.

Any assistance is appreciated.

[rohitarorait82 (Rohit arora)]

Have you done any changes in elasticsearch.yml file ?

Are you able to run elasticsearch from the location where it is actually installed using logstash executable file

Can you share output of systemctl status elasticsearch -l and journalctl -xe

[rcoote (Rob Coote)]

Yes, just what was reflected in the video. Here is my config:

elasticsearch.yml:

cluster.name: Elastiflow

node.name: node1

path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch

bootstrap.memory_lock: true

network.host: 0.0.0.0
http.port: 9200

discovery.type: single-node

indices.query.bool.max_clause_count: 8192
search.max_buckets: 250000

action.destructive_requires_name: true

reindex.remote.whitelist: *:*

xpack.monitoring.enabled: true
xpack.monitoring.collection.enabled: true
xpack.monitoring.collection.interval: 30s

xpack.security.enabled: true
xpack.security.audit.enabled: false

node.ml: false
xpack.ml.enabled: false

xpack.watcher.enabled: false

xpack.ilm.enabled: true

xpack.sql.enabled: true

systemctl status elasticsearch -l output:

elasticsearch.service - Elasticsearch
 Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/elasticsearch.service.d
         └─elasticsearch.conf
 Active: failed (Result: exit-code) since Thu 2020-09-17 02:49:12 UTC; 30s ago
   Docs: https://www.elastic.co
Process: 16292 ExecStart=/usr/share/elasticsearch/bin/systemd-entrypoint -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=1/FAILURE)
   Main PID: 16292 (code=exited, status=1/FAILURE)

Sep 17 02:49:11 oeg-elk-01 systemd-entrypoint[16292]:         at org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:364)
Sep 17 02:49:11 oeg-elk-01 systemd-entrypoint[16292]:         at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:227)
Sep 17 02:49:11 oeg-elk-01 systemd-entrypoint[16292]:         at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingValue.produce(ParserImpl.java:586)
Sep 17 02:49:11 oeg-elk-01 systemd-entrypoint[16292]:         at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:158)
Sep 17 02:49:11 oeg-elk-01 systemd-entrypoint[16292]:         at org.yaml.snakeyaml.parser.ParserImpl.getEvent(ParserImpl.java:168)
Sep 17 02:49:11 oeg-elk-01 systemd-entrypoint[16292]:         at com.fasterxml.jackson.dataformat.yaml.YAMLParser.nextToken(YAMLParser.java:355)
Sep 17 02:49:11 oeg-elk-01 systemd-entrypoint[16292]:         ... 14 more
Sep 17 02:49:12 oeg-elk-01 systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
Sep 17 02:49:12 oeg-elk-01 systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
Sep 17 02:49:12 oeg-elk-01 systemd[1]: Failed to start Elasticsearch.

journalctl -xe output:

Sep 17 02:49:59 oeg-elk-01 multipathd[739]: sda: add missing path
Sep 17 02:49:59 oeg-elk-01 multipathd[739]: sda: failed to get udev uid: Invalid argument
Sep 17 02:49:59 oeg-elk-01 multipathd[739]: sda: failed to get sysfs uid: Invalid argument
Sep 17 02:49:59 oeg-elk-01 multipathd[739]: sda: failed to get sgio uid: No such file or directory
Sep 17 02:50:04 oeg-elk-01 multipathd[739]: sda: add missing path
Sep 17 02:50:04 oeg-elk-01 multipathd[739]: sda: failed to get udev uid: Invalid argument
Sep 17 02:50:04 oeg-elk-01 multipathd[739]: sda: failed to get sysfs uid: Invalid argument
Sep 17 02:50:04 oeg-elk-01 multipathd[739]: sda: failed to get sgio uid: No such file or directory
Sep 17 02:50:09 oeg-elk-01 multipathd[739]: sda: add missing path
Sep 17 02:50:09 oeg-elk-01 multipathd[739]: sda: failed to get udev uid: Invalid argument
Sep 17 02:50:09 oeg-elk-01 multipathd[739]: sda: failed to get sysfs uid: Invalid argument
Sep 17 02:50:09 oeg-elk-01 multipathd[739]: sda: failed to get sgio uid: No such file or directory
Sep 17 02:50:14 oeg-elk-01 multipathd[739]: sda: add missing path
Sep 17 02:50:14 oeg-elk-01 multipathd[739]: sda: failed to get udev uid: Invalid argument
Sep 17 02:50:14 oeg-elk-01 multipathd[739]: sda: failed to get sysfs uid: Invalid argument
Sep 17 02:50:14 oeg-elk-01 multipathd[739]: sda: failed to get sgio uid: No such file or directory
Sep 17 02:50:19 oeg-elk-01 multipathd[739]: sda: add missing path
Sep 17 02:50:19 oeg-elk-01 multipathd[739]: sda: failed to get udev uid: Invalid argument
Sep 17 02:50:19 oeg-elk-01 multipathd[739]: sda: failed to get sysfs uid: Invalid argument
Sep 17 02:50:19 oeg-elk-01 multipathd[739]: sda: failed to get sgio uid: No such file or directory
Sep 17 02:50:24 oeg-elk-01 multipathd[739]: sda: add missing path
Sep 17 02:50:24 oeg-elk-01 multipathd[739]: sda: failed to get udev uid: Invalid argument
Sep 17 02:50:24 oeg-elk-01 multipathd[739]: sda: failed to get sysfs uid: Invalid argument
Sep 17 02:50:24 oeg-elk-01 multipathd[739]: sda: failed to get sgio uid: No such file or directory
Sep 17 02:50:29 oeg-elk-01 multipathd[739]: sda: add missing path
Sep 17 02:50:29 oeg-elk-01 multipathd[739]: sda: failed to get udev uid: Invalid argument
Sep 17 02:50:29 oeg-elk-01 multipathd[739]: sda: failed to get sysfs uid: Invalid argument
Sep 17 02:50:29 oeg-elk-01 multipathd[739]: sda: failed to get sgio uid: No such file or directory
Sep 17 02:50:34 oeg-elk-01 multipathd[739]: sda: add missing path
Sep 17 02:50:34 oeg-elk-01 multipathd[739]: sda: failed to get udev uid: Invalid argument
Sep 17 02:50:34 oeg-elk-01 multipathd[739]: sda: failed to get sysfs uid: Invalid argument
Sep 17 02:50:34 oeg-elk-01 multipathd[739]: sda: failed to get sgio uid: No such file or directory
Sep 17 02:50:39 oeg-elk-01 multipathd[739]: sda: add missing path
Sep 17 02:50:39 oeg-elk-01 multipathd[739]: sda: failed to get udev uid: Invalid argument
Sep 17 02:50:39 oeg-elk-01 multipathd[739]: sda: failed to get sysfs uid: Invalid argument
Sep 17 02:50:39 oeg-elk-01 multipathd[739]: sda: failed to get sgio uid: No such file or directory
Sep 17 02:50:44 oeg-elk-01 multipathd[739]: sda: add missing path
Sep 17 02:50:44 oeg-elk-01 multipathd[739]: sda: failed to get udev uid: Invalid argument
Sep 17 02:50:44 oeg-elk-01 multipathd[739]: sda: failed to get sysfs uid: Invalid argument
Sep 17 02:50:44 oeg-elk-01 multipathd[739]: sda: failed to get sgio uid: No such file or directory
Sep 17 02:50:49 oeg-elk-01 multipathd[739]: sda: add missing path
Sep 17 02:50:49 oeg-elk-01 multipathd[739]: sda: failed to get udev uid: Invalid argument
Sep 17 02:50:49 oeg-elk-01 multipathd[739]: sda: failed to get sysfs uid: Invalid argument
Sep 17 02:50:49 oeg-elk-01 multipathd[739]: sda: failed to get sgio uid: No such file or directory
Sep 17 02:50:54 oeg-elk-01 multipathd[739]: sda: add missing path
Sep 17 02:50:54 oeg-elk-01 multipathd[739]: sda: failed to get udev uid: Invalid argument
Sep 17 02:50:54 oeg-elk-01 multipathd[739]: sda: failed to get sysfs uid: Invalid argument
Sep 17 02:50:54 oeg-elk-01 multipathd[739]: sda: failed to get sgio uid: No such file or directory
Sep 17 02:50:58 oeg-elk-01 sudo[16545]: oegadmin : TTY=pts/0 ; PWD=/home/oegadmin ; USER=root ; COMMAND=/usr/bin/journalctl -xe
Sep 17 02:50:58 oeg-elk-01 sudo[16545]: pam_unix(sudo:session): session opened for user root by oegadmin(uid=0)

[warkolm (Mark Walkom)]

Please format your code/logs/config using the </> button, or markdown style back ticks. It helps to make things easy to read which helps us help you

[Steve_Mushero (ELKman.io (ELK Stack Management Tool))]

Seems YAML format / parse issue; maybe check your YAML file with a verify utility online to make sure it's legal.

reindex.remote.whitelist: *:*

Does that need quoting? I ran a quick test at http://www.yamllint.com/ and yes, you need "." there and the YAML validates, else errors in that validator.

[rcoote (Rob Coote)]

That did it, thank you!