Elasticsearch won't upstart

Elastic Stack Elasticsearch
1

Hi!

I have been successfully running ELK for a day :slight_smile:

However, after a needed server restart I cannot upstart Elasticsearch from systemctl.

If I run it manually it works.

My set up:

centos7
elasticsearch-2.2.1-1.noarch

Running from systemctl gives:

` [root@localhost share]# sudo /bin/systemctl status elasticsearch.service
● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Mon 2016-03-28 01:26:45 BST; 21s ago
Docs: http://www.elastic.co
Process: 3936 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -Des.pidfile=${PID_DIR}/elasticsearch.pid -Des.default.path.home=${ES_HOME} -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_DIR} -Des.default.path.conf=${CONF_DIR} (code=exited, status=127)
Process: 3934 ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec (code=exited, status=0/SUCCESS)
Main PID: 3936 (code=exited, status=127)

Mar 28 01:26:45 localhost.localdomain systemd[1]: Starting Elasticsearch...
Mar 28 01:26:45 localhost.localdomain systemd[1]: Started Elasticsearch.
Mar 28 01:26:45 localhost.localdomain systemd[1]: elasticsearch.service: main process exited, code=exited, status=127/n/a
Mar 28 01:26:45 localhost.localdomain systemd[1]: Unit elasticsearch.service entered failed state.
Mar 28 01:26:45 localhost.localdomain systemd[1]: elasticsearch.service failed.`

All files/folders seem to be in place.

Here is the .service config:

[Unit]
Description=Elasticsearch
Documentation=http://www.elastic.co
Wants=network-online.target
After=network-online.target

[Service]
Environment=ES_HOME=/usr/share/elasticsearch
Environment=CONF_DIR=/etc/elasticsearch
Environment=DATA_DIR=/var/lib/elasticsearch
Environment=LOG_DIR=/var/log/elasticsearch
Environment=PID_DIR=/var/run/elasticsearch
EnvironmentFile=-/etc/sysconfig/elasticsearch

WorkingDirectory=/usr/share/elasticsearch

User=elasticsearch
Group=elasticsearch

ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec

ExecStart=/usr/share/elasticsearch/bin/elasticsearch
-Des.pidfile=${PID_DIR}/elasticsearch.pid
-Des.default.path.home=${ES_HOME}
-Des.default.path.logs=${LOG_DIR}
-Des.default.path.data=${DATA_DIR}
-Des.default.path.conf=${CONF_DIR}
StandardOutput=journal
StandardError=inherit

Specifies the maximum file descriptor number that can be opened by this process

LimitNOFILE=65535

Specifies the maximum number of bytes of memory that may be locked into RAM

Set to "infinity" if you use the 'bootstrap.mlockall: true' option

in elasticsearch.yml and 'MAX_LOCKED_MEMORY=unlimited' in /etc/sysconfig/elasticsearch

#LimitMEMLOCK=infinity

Disable timeout logic and wait until process is stopped

TimeoutStopSec=0

SIGTERM signal is used to stop the Java process

KillSignal=SIGTERM

Java process is never killed

SendSIGKILL=no

When a JVM receives a SIGTERM signal it exits with code 143

SuccessExitStatus=143

[Install]
WantedBy=multi-user.target

Built for Distribution: RPM-2.2.1 (rpm)`

elasticsearch user is owner of all the above mentioned folders.. I'm not sure where to go.

Any help will be much appreciated!

1 Like
2

What do your ES logs show?

3

Actually I'm not able to find any related logs when is start with systemctl. Based on the config file I expect to find them:

/var/log/elasticsearch/

But nothing is going there anymore. When I start it up manually the logs go into:

/usr/share/elasticsearch/logs/elastic.log

4

I believe I'm ending in similar situation on CentOS Linux release 7.2.1511 (Core) with 5.0.0_alpha4-1.

[root@lab elasticsearch]# service elasticsearch status
● elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Wed 2016-07-06 18:16:35 CEST; 7min ago
     Docs: http://www.elastic.co
  Process: 45928 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid -Edefault.path.logs=${LOG_DIR} -Edefault.path.data=${DATA_DIR} -Edefault.path.conf=${CONF_DIR} (code=exited, status=1/FAILURE)
  Process: 45926 ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec (code=exited, status=0/SUCCESS)
 Main PID: 45928 (code=exited, status=1/FAILURE)

Jul 06 18:16:35 lab systemd[1]: Starting Elasticsearch...
Jul 06 18:16:35 lab systemd[1]: Started Elasticsearch.
Jul 06 18:16:35 lab systemd[1]: elasticsearch.service: main process exited, code=exited, status=1/FAILURE
Jul 06 18:16:35 lab systemd[1]: Unit elasticsearch.service entered failed state.
Jul 06 18:16:35 lab systemd[1]: elasticsearch.service failed.

Main problem is that there was no log produced, so I'm not sure how to debug it further.

1 Like
5

I also ran into the same situation.
I was using ES 5.0 on CentOS 7.
I downgraded it to CentOS 6 and now it's working fine.

6

Get same issue here, I am restarting it and then it stucked there. There is no elasticsearch proces but I cannot start it successfully.

8

Hi , I have faced the same problem in RHEL where the elasticsearch won't generate logs .
Then The problem was due to an extra space in the elasticsearch.yml file in /etc/elasticsearch path.